If my State is being negligent in its responsibilities under the Chicago Convention,
can ICAO reprimand or penalize it in some manner?
Only national governments can consider holding other countries to account.
That accountability normally takes the form of country-to-country sanctions which are assessed and agreed to through higher-level multilateral platforms than ICAO, for example the UN General Assembly or UN Security Council.
The international aviation convention which established ICAO prioritizes standardization and cooperation among its 193 member countries, as negotiated and endorsed through the assemblies, committees, working groups, etc., which governments mandate and budget our secretariat to organize for them.