Construction crews have gotten to work on what was at one point a salon and barber shop on Park Heights Avenue.
"It just doesn't make sense to me and we don't want it. We're not going to have it," George Mitchell, President of Neighborhoods United, said.
Mitchell said he learned the building will be turned into a methadone clinic. It's a proposal Mitchell and the Pimlico Merchants Association are banding together to stop.
"Everybody is in an uproar. No one has asked the question. No one came to seek or ask the question with the community, with the merchants, with the community leaders. This is going to be here. We just happened to find out," said Sister Yeshiyah Israel, Vice President of Pimlico Merchant Association.
The building sits in the 5200 block of Park Heights Avenue. It is right across from Pimlico Race Track.
The group said that is right in the downtown area of the community and it is one they have been working to revitalize.
Danita Maryland is the director at the ABC Child Development Center, which sits a few doors down.
"Nobody came to me and said, 'Hey, we're looking to put a methadone clinic there.' I understand that it's not their responsibility to do that but it was a common courtesy, dealing with kids in the community and being here. When are they going to think about the kids?" Maryland asked.
"My kids are going to be here as long as I know it's safe. There haven't been any problems and honestly with the clinic, I think they should just be discrete. That's all. ... I'm just concerned. I'm not against it. I'm not opposed to it. I wouldn't even vote against it," Antione Mercer, a father who brings his son to the ABC Center, said.
Community leaders said that is not enough and they have already called their representatives. They told ABC2 the plan is to organize and mobilize to stop the plans from moving forward.
"They act like people don't have children, people don't care. I tell you if they want to open up a methadone clinic, whoever approved this, open it up next door to your house and see how you feel," Mitchell said.