In your grid properties first make sure to choose EPSG:4326 as CRS, then choose "custom" as coordinate format and click on the expression button next to it:
Now use this expression:
abs(@grid_number) || '°' || -- concat absolute grid number (without -) with °
if(@grid_number=0,'', -- if grid number is 0, dont use a suffix
if(
@grid_axis = 'x', -- determine which grid axis it is
if(@grid_number<0,'W','E'), -- If grid number is negative and axis is x, use W, else E as suffix
if(@grid_number<0,'S','N') -- If grid number is negative and axis is y, use S, else N as suffix
)
)
Result: