[Legalism] means treating Biblical standards of conduct as regulations to be kept by our own power in order to earn God’s favor. In other words, legalism is present whenever a person is trying to be ethical in his own strength, that is, without relying on the merciful help of God in Christ. Simply put, moral behavior that is not from faith is legalism (Rom. 14:23).
The legalist is usually a moral person. In fact, the majority of moral people in the Western world are legalists because their so-called Judeo-Christian morality, inherited from their forefathers, does not grow out of a humble, contrite reliance on the blood-bought, Spirit-wrought, merciful enabling of God. On the contrary, for the legalist, morality serves the same function that immorality does for the antinomian or the progressive–namely, as the expression or self-reliance and self-assertion. The reason some Pharisees tithed and fasted was the same reason some university students take off their clothes and lie around naked in the parks in Munich and Amsterdam.
The moral legalist is the elder brother of the immoral prodigal (Luke 15:11-32). They are blood brothers in God’s sight because both reject the mercy of God in Christ as a means to righteousness and use either morality or immorality as a means of expressing their independence and self-sufficiency and self-determination. And it is clear from the New Testament that both will result in tragic loss of eternal life, if there is no repentance.
John Piper, Brothers, We Are Not Professionals: A Plea to Pastors for Radical Ministry, p. 153