Many workers have been killed - many more than the official death toll of 14, a Greek union representative said.A BBC team visited a variety of sites in Greece and reported that similar site would have been shut down had employees been working under the same conditions in Britain.
But, he told the programme, no official numbers are being kept.
The BBC team observed workers - many from Eastern Europe and South Asia - operating without protective equipment and with minimal supervision. Speaking on BBC Radio 4's Face the Facts programme, Greek Olympic Committee President Lampis Nikolaou admitted that the death toll in Athens was far greater than in any other city to have recently hosted the games.
The general secretary of the Greek Construction Workers' Union, George Theodorou, told the programme he had collected names and details of 14 workers who had died on Olympic facilities.
But he believes there have been many more deaths on all the supporting infrastructure, like Athens' new roads, tram lines and metro, taking the actual death toll to 40.
"Men are being forced to work long shifts, up to 14 hours a day every day, in very hot temperatures and under constant pressure to complete construction work in time for the Olympics," he said.
"Most have no hard hats or safety boots and if they complain, they're sacked."