Legalists sometimes defend themselves by claiming that legalism, properly understood, is just what Paul condemned in Galatians 1: the sin of making justification conditional on some work or ceremony performed by the sinner. In other words, legalism is works-salvation. So, they say, if you formally affirm the principle of sola fide and preach that people can be saved without any prerequisite work, you can't possibly be a legalist, no matter how many rules you make and impose on the consciences of people who are already converted.
No. Legalism is the error of abandoning our liberty in Christ in order to take on a yoke of legal bondage (Galatians 5:1). There are actually two kinds of legalism.
First is the one recognized and despised even by the fundamentalist with his thick rule-book. It's the legalism of the Judaizers. The Judaizers wanted to make circumcision a requirement for salvation. They had fatally corrupted the gospel by adding a human work as a requirement for salvation. That is certainly the worst variety of legalism, because it destroys the doctrine of justification by faith and thereby sets up "a gospel contrary to the one you received" (Galatians 1:8-9).
But another kind of legalism is the legalism of the Pharisees. It's the tendency to reduce every believer's duty to a list of rules. This is the kind of legalism that often seems to surface in our comment-threads. At its root is a belief that holiness is achieved by legal means—by following a list of "standards." This type of legalism doesn't necessarily destroy the doctrine of justification like the legalism of the Judaizers. But it does destroy the doctrine of sanctification, and it is certainly appropriate to call it what it is: legalism—i.e., a sinful misapplication of law; an attempt to make law do work that only grace can do. Like the Judaizers' brand of legalism, it brings people under a yoke of bondage Scripture has not placed on them.
As a matter of fact, that is exactly what Jesus said about the legalism of the Pharisees: "They tie up heavy burdens, hard to bear, and lay them on people's shoulders" (Matthew 23:4).
Pharisaical legalists are not content to live life in the power of the Spirit, cultivate discernment, and avoid things that are clearly profane or immoral; they make lists of rules that prohibit Christians from practically everything but church activities. It's not enough to avoid gambling; they insist that good Christians will avoid card-playing altogether. They're not content with doing things in moderation and with self-control, they make rules that call for strict abstinence from everything doubtful—and they try to impose those rules on other Christians—saddling people with a yoke that they imagine exists somewhere in the white spaces of Scripture.
You want rules? Here's a good one to start with: When it comes to the question of spiritual duties, where Scripture stops speaking, we should, too.