I am searching for spatial statistics software which has many powerful features and supports a GIS interface.
Can anyone give me some advice?
This question has been converted to Community Wiki and wiki locked because it is an example of a question that seeks a list of answers and appears to be popular enough to protect it from closure. It should be treated as a special case and should not be viewed as the type of question that is encouraged on this, or any Stack Exchange site, but if you wish to contribute more content to it then feel free to do so by editing this answer.
In a nutshell: Start with QGIS.
There are several Free and Open Source tools for geospatial statistics.
There are several spatial statistics plugins in Quantum GIS, such as
Linfinity has provided a brief tutorial covering QGIS Spatial Stastics.
Anita Graser has written a tutorial on the QGIS Group Statistics plugin.
QGIS 2.0 Processing is a graphical modelling environment that integrates with several prominent projects, including SAGA and GRASS, some of which contain various statistical algorithms.
SAGA has over 400 modules for geoprocessing and statistical analysis.
GRASS contains several zonal statistics functions.
The R Language has several spatial extensions.
The Spatstat module for R provides a host of spatial statistics functions.
R Studio is a very nice IDE for the R Language, and will help you to easily locate and install the R spatial libraries.
GeoDa is free, cross-platform software designed for dynamic visualization, exploratory spatial data analysis, and spatial statistics. It has been around for almost 15 years (starting as an ArcView 3.x extension, it was recoded to be independent of ArcView after ESRI abandoned the old AV architecture). It is associated with an illustrious group of GIS educators and researchers.
See also GRASS and R integration:
http://grass.osgeo.org/wiki/R_statistics
For example boxplots:
... or decision trees (rpart example):
ArcGIS has Spatial Statistics Toolbox for statistical information. it analyze spatial distributions, patterns, processes, and relationships.
Spatial statistics allow you to:
Summarize the key characteristics of a distribution. Identify statistically significant spatial clusters (hot spots/cold spots) and spatial outliers. Assess overall patterns of clustering or dispersion. Model spatial relationships.
you can get information here.
A new software that is now also available is insensa GIS at www.insensa.org, "free and open source software for statistical computing and display of GIS data." You can also e.g. calculate correlations, view scatterplots ... In case the function you want to use is not available, you may also write your own plugin.
There are few interesting packages from the Spatial Data Mining and Visual Analytics Lab, especially:
S4 research group also has few useful tools, especially:
CrimeStat is also worth having a look. Although focused on crime analysis, the functionality can be adapted to other purposes.
SANET is an ArcGIS toolset for analysis based on networks.
GeoSOM is a package for creating spatial Self-Organizing Maps.
Although lacking any mapping capabilities SaTScan is an essential tool for exploration of spatial and spatio-temporal clusters. Fast Spatial Scan from The Auton Lab can cope with larger datasets.