"Garlic mustard is an invasive non-native biennial herb that spreads by seed. It is difficult to control once it has reached a site; it can cross-pollinate or self-pollinate, it has a high seed production rate, it out competes native vegetation and it can establish in a relatively stable forest understory. It can grow in dense shade or sunny sites. The fact that it is self-fertile means that one plant can occupy a site and produce a seed bank. Plant stands can produce more than 62,000 seeds per square meter to quickly out compete local flora, changing the structure of plant communities on the forest floor." (source: here.) In otter words, it will completely take over everything. It is like a cancer. It is like The Little Prince's Baobabs, the roots of which threatened to break apart his tiny planet. It replicates itself inexorably like the Orcs in The Lord of the Rings that destroyed everything in their path. So, what to do? Here is a tutorial for your enjoyment.
Go out when the soil is crumbly and semi-moist, but not wet. Look for these "rosettes" as shown below- this is the first year plant (remember, it's a biennial- the first and second-year plants look different). The first-year plant is much easier to remove, having a less-developed root system.
Pull the plant gently but firmly, from the base where the stem and root meet. 
Make sure you are getting the whole root. The taproot, like a dandelion or a thistle (or a pimple, for that matter), must be completely removed or it will sprout anew. The root is "allelopathic," meaning it produces chemicals that inhibit the growth of tree seedlings and other plants. Garlic mustard is a playground bully, all around.
Mission accomplished! Janine from Gracious Gardens tells me that the "seed bank"- the seemingly inexhaustible supply of invasive weed seeds- remains stealthily waiting in the soil for up to seven years.
The seeds are tiny - remember the Biblical parable about "faith as small as a mustard seed?" Well then. You need to be careful to disturb the infested soil as little as possible due to the tiny seeds. Double bag the plants and make sure the bags do not have any holes. (Remember the seeds are tiny!!! They can be carried in clothing, shoes or mud. Be sure to clean shoes, pockets, cuffs and equipment after removing and bagging the plants. Dispose of plants in the trash, or burn. You do not want to compost this plant.
The seeds are tiny - remember the Biblical parable about "faith as small as a mustard seed?" Well then. You need to be careful to disturb the infested soil as little as possible due to the tiny seeds. Double bag the plants and make sure the bags do not have any holes. (Remember the seeds are tiny!!! They can be carried in clothing, shoes or mud. Be sure to clean shoes, pockets, cuffs and equipment after removing and bagging the plants. Dispose of plants in the trash, or burn. You do not want to compost this plant.
Stay vigilant, and repeat as necessary.
For more details, including managment with pesticides, see here.
