The article I first linked to in the Q about adding Georss was the wrong one, it's about adding georss to Arcgis Desktop via the Data Interoperability Extension. (However georss feeds are not supported in Data Interop base edition, which effectively means that Arcgis Desktop does not have georss support.)
I cross posted a similar question to the official Arcgis Online forum, and have received zero replies to date. I've also opened a technical support case for same. No usable information from them yet, but it's only been a day.
I'll update this answer as needed, but my provisional conclusion is: you can't, Arcgis Online only exports pretty pictures in the browser.
Update: The provisional conclusion was half correct: the free Arcgis Online has no export options, however one of the features with subscription based Arcgis Online for Organizations is hosted Feature Services, and those can be exported as CSV or shapefile. There is still no georss/json, so even with a subscription there still isn't a way to easily re-use your features in other ways (we want to have single data stream of coords, story, and photo that feeds a blog as well as the web map), but at least the data is not completely imprisoned.
Thanks go to Mike Minami of Esri for telling me about the subscription feature services feature, here.
March-2013: see How to convert ArcGIS Server JSON to GeoJSON?. ogr2ogr, can be used to convert from the ArcGIS Online (or Server) REST API to both geojson and georss.