To explore efficient improvement modes for acidified soils, a field experiment was conducted in acidified paddy fields of Duchang County, Jiangxi Province. Based on conventional fertilizer application, five combination treatments were implemented including the addition of lime, Chinese milk vetch (Astragalus sinicus) green manure, and organic fertilizer. Soil acidification indicators, nutrient contents, and aggregate structure were evaluated. The results show that, compared with the conventional control, all treatments increase rice yield and significantly improve soil pH. The addition of Chinese milk vetch green manure on the basis of lime application significantly enhances soil available phosphorus content. The combination of conventional fertilizer + lime + organic fertilizer significantly increases the proportion of large soil aggregates. The treatment with a 30% reduction in chemical nitrogen fertilizer combined with lime, green manure, and organic fertilizer significantly raises soil pH to 5.01, reduces exchangeable aluminum by 19.7%, and increases the contents of alkali-hydrolyzable nitrogen, available phosphorus, and available potassium by 27.6%, 152%, and 1.87%, respectively. Integrating reduced chemical fertilization with organic fertilizer, green manure, and lime demonstrates superior comprehensive performance in ameliorating acidified soils and enhancing rice yield, and provides a valuable reference for the improvement of similar acidified farmlands.