I have a shapefile with a lot of polygons in it.
These polygons are grouped into a category: district.
How can I merge the polygons based on the same name of the category within this shapefile?
I believe the tool you are looking for is the Dissolve tool.
You can specify which field you want to dissolve your feature on, so in your case you would specify "district" and it will dissolve all of your polygons based on the value in that field.
QGIS has a tool to dissolve polygons. Look into Vector -> GeoProcessingTools->Dissolve
If you don't want to alter your actual shapefile/feature class, you can use symbol levels to make your layers appear dissolved by attributes. It takes some getting use to, but play around with it and I suspect you'll eventually get symbology you want without having to alter your actual data.