It depends on what you are trying to do. Octave is usually compared to Matlab, not SPSS.
Disclaimer: I have never used Octave.
Octave does have support for some basic statistical functions. The Octave Manual has more information. Octave itself does not provide a GUI. It is a command-line program. There is however an interesting project called QT-Octave which appears to be an attempt of creating a matlab like interface on top of Octave. If you need to work with numerical analysis only, this could be a good solution for you. Octave 3.0.1 and qtoctave 0.7.4 are both in the Ubuntu repos, making them easy to install.
Unless you are an engineer or are otherwise dealing with matrices only, Octave may be a little limiting, especially when compared to a tool such as SPSS. Fortunately, there are other options! If you search this forum you can find plenty of threads discussing the pros and cons of the software I am about to suggest:
Format: Title, License, Notes
- R, OSS, The 800 pound gorilla of open-source statistics
- PSPP, OSS, Incomplete but excellent clone of SPSS
- SPSS, Proprietary, Available for Linux too!
- SAS, Proprietary, The "other" proprietary stats package.
- Stata, Proprietary, Ask someone else. I don't know nothing about it.
I've included links to R and PSPP. You can find the others on your own. R and PSPP are BOTH excellent pieces of software. Without knowing what you want to do, I can't really recommend one over the other. R is probably the most advanced, comprehensive statistics package ever created. It is also probably over-kill for a highschool or under-grad level stats course. PSPP comes with a user friendly GUI, but is not a complete product. SPSS currently provides more functionality than PSPP.
If you are a student, SPSS will sell you a version of SPSS that runs on Linux for a very "reasonable" price compared to the commercial version. As long as your data sets are fairly small, this will probably get you through a stats class. As an added benefit, the SPSS syntax you learn will work on PSPP, provided the functionality has been implemented in the latter.
I've never used SAS or Stata but I know there are threads here that discuss installing them on Ubuntu. I assume that with effort they can be made to run effectively. People seem to like them, but I don't know much about them one way or the other.
Look this over and tell us what you need to do and how much computer / programming experience you have and I can try to give you a more definite recommendation. You might also consider renaming this thread. There are other people in this forum with experience doing statistical analysis. Getting their input could be beneficial.