You can use data defined properties for this. Use the style tab on the layer properties and click the expression button to the right of the color.
In there you can use the function
color_rgba( red, green, blue, alpha ) to create the color. All values need to be between 0 and 255.
color_rgba( 255, 0, 0, ( 1 - "transparency" ) * 255 )
This will give you a totally red style with the alpha defined from a field "transparency" as per your specification in the question.
In the expression editor you will find a number of other color related functions that may better suit your needs, just explore them and read their documentation directly in the expression editor.
Another example for nominal (qualitative) values which you would normally classify is to work in the hsv space:
There is the function
color_hsva( hue, saturation, value, alpha ) to create the color. The values need to be between
- hue: 0-360 (meaning see below)
- value and saturation: 0-100
- alpha: 0-255.
WHEN "classification" = 'red' THEN 0
WHEN "classification" = 'blue' THEN 240
END CASE, -- hue
80, -- saturation
80, -- value
( 1 - "transparency" ) * 255 -- alpha
For QGIS >= 2.12 also consider @ndawsons answer.