Use RGB mapping! Found this helpful info at the link below.
Create a CSV file that contains an RGB color associated with each polygon. In the file you'll have an ID column (this should be the same in the polygon and the point shapefile so you can link them) and 3 additional columns for "R", "G",
"B". Load this into QGIS and join the CSV to your polygon and point file.
Then in QGIS, open Layer Properties >> Style Tab, and make sure Single Symbol is selected. Click the box next to the "Fill" color and choose "Edit.
You will be editing the base color of your polygon bases on RGB colors present on the data table, with a constant border color (you can apply the same criteria if needed).
After hitting "Edit" just put the 3 RGB columns names of your table using the following expression:
Reference: Using field-to-RGB mapping for symbology in QGIS?