This is not case sensitive.
Step 1
I have two shapefiles. Circles and Squares. Both have an id
column/field and both have an address
column/field.

Step 2
Use the Join Attributes by Field Values
tool. For the "Layer 2 fields to copy", select the other shapefiles address column. You can choose to "discard the records which could not be joined" as well. You're call. Save and Run.

Step 3
Open the attribute table of the newly created layer and see that they joined based on the id
field.

step 4
Create a new field and make sure the "Output length" is set high enough for all characters to be present from the concatenation of addresses. Then perform the operation you want. Here I combined the address separated by an --
.

Step 5
Final Results

EDITS
You are correct in thinking that the text needs to match (case sensitive).
Update your address field to all CAPS. Reference @hugonbg answer.
Perform the same steps indicated below expect use the address field as your "Table field" parameter indicated in "Step 2".
id
column in both shapefiles and joining based on that. Would it be possible to share your shapefiles?