A unanimous vote at a crowded residents' meeting has called for opposition to a development at the back of Potters Field.
There were also calls to extend Potters Fields Park on to the coach park rather than have development on the entire site.
The Council's delay in considering the plan by Berkeley Homes for tower blocks, including a 19 storey building, has resulted in a public inquiry in April.
Lisa O'Donnell of Southwark Planning said that there were at least twelve reasons for refusing the plan including the sensitive setting alongside Tower Bridge and opposite the World Heritage Site embracing the Tower of London.
The Council, which owns part of the land, prefers an arts or cultural focus for the site.
Fears were expressed over the tower block scheme blocking sunlight from the riverside park which is being redesigned at a cost of more than £4m.
The Association agreed to form a steering committee to prepare evidence for the inquiry. Among residents supporting an alternative to the Berkeley Homes scheme are architect Nicholas Lacey and writer Jonathan Meades.