I have a shapefile with a lot of polygons in it.
These polygons are grouped into a category: district.
How can I merge the polygons based on the same name of the category within this shapefile?
I believe the tool you are looking for is the Dissolve tool.
You can specify which field you want to dissolve your feature on, so in your case you would specify "district" and it will dissolve all of your polygons based on the value in that field.
QGIS has a tool to dissolve polygons. Look into Vector -> GeoProcessingTools->Dissolve