Use the Buffer Tool with your points and then use the Spatial Join tool. For the Spatial Join make the point buffers the target and the polygons the join features with the One To One option and change the merge property of the column with the values you want averaged in the field list to use Mean instead of Sum (or add a new field to the field list if you want both the sum of the values and the mean in separate fields). The point buffers and Spatial Join output will retain the FID of the original points in a field, so you can transfer the Mean from the Spatial Join output into the original points using a standard Join and the Field Calculator.
Technically you do not have to do the Buffer, since you can set a radius/tolerance for the Spatial Join tool to function like a buffer. However, the Buffer tool is fast and the Spatial Join tool will run much faster when you use a radius/tolerance of 0.