The plumber typically checks the burner or elements, thermostat, relief valve, venting, and the tank's age and condition, then tells you whether a repair makes sense. In older Hingham houses, they'll also look at the flue or chimney connection and the space around the heater, since replacement units don't always fit where the old one stood.
Water heater replacements generally need a plumbing permit, and gas units a gas permit, through the Building Department at Town Hall. Its Plumbing Inspector books inspections by phone on weekday mornings, Monday through Thursday, and your plumber should arrange that.
If you're moving away from oil or adding propane, ask early whether Hingham Fire Prevention needs to be involved. Hingham Fire Prevention also requires a carbon monoxide detector on every floor when it inspects detectors, which is a good standard to meet whenever fuel-burning equipment is replaced.