To investigate the temporal variation of the sediment prokaryote communities and their relation with the rapid increase of algae population in Taihu, a shallow eutrophic freshwater Lake, water and surface sediments were sampled from seven sites in different stages of algal bloom. The physicochemical characterization revealed positive correlations among the nutrient (N, P) parameters in the water and sediments, as well as TN/TP ratio 30.79 in average in water and 0.13 in sediments, demonstrating that P content was the limit factor for bloom in Taihu and sediment was an important nutrient resource for the water body. T-RFLP analysis of 16S rRNA genes revealed a diversity decrease of sediment prokaryotic communities along the bloom. The bacterial communities in sediments were more sensitive and shaped by the temporal changes, while archaea were more sensitive to the trophic level. The pyrosequencing data showed clear spatial and temporal changes in diversity of sediment bacteria. Betaproteobacteria was the most abundant group in all the samples, following by Delta-, Gama- and Alpha-proteobacteria, Acidobacteria, Chlorobi, Chloroflexi etc. At the genus level, Thiobacillus and Sulfuricurvum were the most dominant groups in the sediments, and the increase of Thiobacillus abundance in February might be used as bioindicator for the subsequent bloom. In addition, NO3 (-)-N was evidenced to be the main factor to regulate the bacterial community structure in the sediments. These results offered some novel and important data for the evaluation and predict the algal bloom in Taihu and can be reference for other shallow fresh water lakes.