The quantum phase transitions and exotic excitations are exciting and important topics of nowadays condensed matter theory. Topologically protected excitations are of great interest for potential applications in quantum computing. This Thesis explores two examples of exotic topologically protected excitations, Majorana fermions and parafermions in hybrid superconductor/semiconductor systems.

In the first part of the thesis, after a brief review of ideas on Majorana zero modes in solid state systems obtained by researchers over the past decade, I present our study of the emergence of Majorana fermions in charge carrier holes doped quantum wires. Study of Majorana modes in this system requires understanding Luttinger holes in low dimensions, which is also crucial for numerous spin-dependent phenomena, emerging field of spintronics and nanotechnology. We find that hole-doped quantum wires that are proximity coupled to a conventional s-wave superconductor is a promising system for the observation of Majorana fermions. We advanced understanding of Luttinger holes in quantum wells and quantum wires. We have shown that the vast majority of earlier treatments of Luttinger holes ignored an important effect, a mutual transformation of heavy and light holes at the heteroboundaries. We have derived the effective hole Hamiltonians in the ground size-quantized sub-bands of quantum wells and quantum wires. The effect of mutual transformation of holes is crucial for understanding Zeeman and spin-orbit coupling, and results in several spin-orbit terms linear in momentum in hole-doped quantum wires. We discuss the criterion for realizing Majorana modes in charge carrier hole systems and show that GaAs or InSb hole wires shall exhibit stronger topological superconducting pairing, providing additional opportunities for its control compared to intensively studies InSb and InAs electron systems.

In the second part of the thesis, I first introduce the basic facts of the current theoretical understanding of the fractional quantum Hall effect and a theoretical model of parafermion excitations. Parafermion zero modes are promising for universal quantum computing. However, physical systems that are predicted to host these exotic excitations are rare and difficult to realize in experiments. I present our work on modeling domain walls on the boundary between gate-induced polarized and unpolarized domains of the fractional quantum Hall effect system near the spin transitions, and the emergence of the parafermion zero modes when such domain wall is proximity coupled to an s-wave superconductor. Exact diagonalization of the Hamiltonian in a disk and torus geometries proves formation of the counter-propagating edge states with different spin polarizations at the boundaries between areas of the electron liquid in polarized and unpolarized filling factor $\nu=2/3$ phases. By analytical and numerical methods we find the conditions for emergence of parafermion zero modes in hybrid fractional quantum Hall/s-wave superconductor system. The phase diagram indicates that the parafermionic phase, which is represented by the six-fold ground state degeneracy, is separated from other phases by a topological phase transition. Such parafermion modes are experimentally feasible. They present a vital step toward the realization of Fibonacci anyons that allow a full universal set of quantum operations with topologically protected quasiparticles.