Redevelopment of the Elizabeth House site in Waterloo's York Road could be completed as soon as 2015.
The site, opposite the Shell Centre in York Road, was recently purchased jointly by Chelsfield and London & Regional Properties after a plan by P&O for three towers was vetoed by former communities secretary John Denham.
The new owners have appointed David Chipperfield Architects to devise a new office-led scheme.
Billy Prendergast, speaking for the architectural team, outlined the initial ideas on how to develop the key site and the priorities being considered.
The many influences for possibilities under the offices include Madrid's Atocha Station winter garden and the fish market in Venice. Retail units might be kept to a minimum or even excluded.
"The current Elizabeth House has contributed to the not so glorious image of Waterloo," said Chelsfield's project director Yair Ginor who invited the audience to share ideas.
"We are confident that the new Elizabeth House will be key in transforming that image and creating something magnificent that will benefit not only Waterloo but London in general."
He added: "The site is very complex from structure to public realm. It's a huge challenge."
Several residents raised questions about the possible position of any tower and its effect on views from both sides of the River Thames including Roupell Street. The P&O scheme included three towers quickly dubbed the 'ugly sisters'.
If the entire development is given over to office accommodation there could be as many as 10,000 new employees in Waterloo.
Chelsfield hopes to seek planning permission next April with the intention of starting construction in 2012.