Wow, mine was set on Remove from current selection the whole time. That was weird. Thank you!