Assuming that you have a polygon layer named 'polygonsA', with a field named 'idA';
and you want to make the difference between any feature of that layer and the feature of the polygon layer 'polygonsB' that have the same (idA) value in the field 'idB'.
It is possible in a simple way as long as there is a single feature in polygonsB for each value of the attribute. If not, Dissolve first the polygonsB layer by the idB field.
Open the Geometry by expression tool.
Select polygonsA as input layer.
Select Polygon as output geometry type.
Write the following geometry expression:
difference( $geometry, geometry( get_feature( 'polygonsB', 'idB', attribute( $currentfeature, 'idA'))))
The output is a polygon layer named 'Modified geometry' with the difference by attribute between polygonsA and polygonsB.