The talk will focus on Monte Carlo and quasi-Monte Carlo methods for Linear Algebra problems. Serial Monte Carlo methods have come to existence in the late 1950´s with various degrees of advances during the years. We have been observing increased interest in these methods in the late 1990´s - 2000´s and with the advent of extreme scale computing a lot of new algorithmic developments were made due to inherent parallelism of Monte Carlo methods. The author will present the latest advances of parallel Monte Carlo methods for Matrix Inversion and solving Systems of Linear Algebraic Equations as well as fast hybrid Monte Carlo and quasi-Monte Carlo for solving Systems of Linear Algebraic Equations. Computational experiments on various advanced parallel computer architectures will be presented.