I would say that the most used Math libraries are Intel Math Kernel Library (Intel MKL), AMD Core Math Library (ACML), and in particular of BLAS there is the GOTO one.
Intel MKL latest release is the 10.3 release. In includes support for BLAS, LAPACK, Trust Region Solver, ScaLAPACK and Cluster FFT. If you are using Intel CPUs, MKL is a library that you would like to use. Mathematical domains supported by Intel MKL are sparse Linear Algebra—sparse BLAS, sparse format converters, PARDISO direct sparse solver, iterative sparse solvers and pre-conditioners; fast fourier transforms, LINPACK benchmark, vector math library and statistics functions - vector random number generators, summary statistics library. The first place to start is http://software.intel.com/en-us/articles/intel-mkl/.
If you have AMD based systems, AMD ACML can provide higher performance. ACML provides a free set of math routines. ACML consists of the following main components: A full implementation of Level 1, 2 and 3 basic linear algebra subroutines (BLAS), a full suite of linear algebra (LAPACK) routines, a suite of fast fourier transforms (FFTs) in both single, double, single-complex and double-complex data types and a random number generators in both single- and double-precision. For downloading ACML (latest version is 5.0) - http://developer.amd.com/libraries/acml/downloads/pages/default.aspx.
For BLAS, GOTO was one of the leading solutions. You can find the latest release on TACC web - http://www.tacc.utexas.edu/tacc-projects/gotoblas2/. GOTO has moved from TACC to Microsoft and it seems that unfortunately he stopped supporting or developing the GOTO BLAS library. If you have any news on this front, drop a comment.