This annoying issue occurs because libgdal was forked between libgdal1 and libgdal1h last year. Ubuntu GIS stable uses libgdal or libgdal1, Ubuntugis unstable uses libgdal1h.
GDAL is a translator library for raster geospatial data formats.
As a library, it presents a single abstract data model to the
calling application for all supported formats. The related OGR
library (which lives within the GDAL source tree) provides
a similar capability for simple features vector data.
You will have to start from fresh.
Make sure you uninstall all software remnants of Ubuntugis Stable / Unstable. Uninstalling libgdal, libgdal1, libgdal1h will remove all software that depends on it. Then check your repos, make sure Ubuntugis Stable & Unstable isn't there.
Then add these repos for QGIS 2.2, Saga 2.1 and Postgres 9.3.3 & PostGIS 2.1.1.
http://www.qgis.org/en/site/forusers/alldownloads.html#ubuntu
Johan Van de Wauw's saga-gis repository
https://wiki.postgresql.org/wiki/Apt
I usually install QGIS first then SAGA, Grass, PostgreSQL 9.3.3, PostGIS 2.1.1, postgresql-contrib-9.3 and finally pgAdmin3.
There are a few small issues. This will install Grass 6.4.3-2, which might not work from Processing in QGIS 2.2. Grass 6.4.3-3, provided by Ubuntugis Unstable, is supposed to fix the problem (but did not work for me on Ubuntu 13.10).
Also, only Ubuntu GIS Unstable provides pgRouting as far as I know (with PostgreSQL 9.1 & PostGIS 2.1.0). If you want pgRouting you will have to compile it from source, it only takes few minutes. If Configure complain that Postgresql-9.1 is not installed, install postgresql-server-dev-9.1 and try again.