NEW ORLEANS –  The civil war that has broken out in the dispute over the fate of three Confederate monuments raged on today under the folded arms of Gen. Robert E. Lee.

Sunday’s protests began with “Take ’em Down NOLA,” the group that loudly supports the monuments removal of four Confederate-era monuments across New Orleans, gathering at Congo Square in the French Quarter.

Congo Square is said to be the historic heart of the African-American experience of slavery in New Orleans– the only place where slaves were allowed to congregate freely for a few hours each week– before the Civil War.

Led by long-time civil rights activist Malcolm Suber, “Take ’em Down NOLA” set out on foot from Congo Square, heading for Lee Circle, about two miles away. It was a diverse group- young and old, black and white- most of them wearing shorts and hats and carrying bottles of water, as if they were heading into the gates at Jazz Fest.

Awaiting their arrival at Lee Circle were dozens of their foes– mostly white men and many from out of state–carrying the Confederate flag and a white flag with a black “x,” – the flag of the movement they called “southern nationalism.” These men were dressed in black and wore motorcycle and baseball helmets as if they were ready for a fight.

One of the men, Michael Hill, from Killen, Alabama, said he was the leader of “League of the South.”

Hill said about 50 members of his group were there “to stop those who would culturally cleanse the South” of its heritage. But Hill said his men did not come to do battle.

“We’re not going to start anything,” said Hill, unless his group felt the need to “defend themselves” instead of the stone General. Hill said his group did not come armed, and he said the members “normally do not heckle” those who hold opposing views. That resolve would be tested.

Nearly four hours after the “Take ’em Down NOLA” group set out from Congo Square, they arrived at Lee Circle and advanced to meet the monument defenders. New Orleans police officers and heavily armed state police in bulletproof vests were there too, and they had set up barricades to keep the two groups apart.

Separated by the steps that lead from a small mound beneath Lee’s statue down to Howard Avenue, the two groups taunted each other, standing just 10 feet apart while the police officers stood between them, silent but watchful.

It appeared that the monument defenders were outnumbered. The ranks of the “take-em-down” group had swelled to at least a couple hundred activists, pushing against the barricades and chanting slogans like, “No hate, no KKK, no racist USA!”

On the other side, Hill’s men from the League of the South stayed mostly quiet. But when one anti-monument activist shook his fist at a monument defender, yelling “look at this!” a monument defender shouted back that the guy with the fist was brave- because he was safe.

“What are you talking sh– for? Look at these barricades,” jeered the monument defender, before walking away in disgust.

In the end, nearing dusk, as voices faltered and feet hurt, both sides drifted away, still hurling an occasional insult as they left.

It was a skirmish at Lee Circle on Sunday, not a battle.

But it will not be either side’s last stand.