Automated speech recognition is one of the more prominent areas of application for computer science activities and is being utilized to a larger extent in recent times. Major advances in machine learning techniques have brought those changes, including advances in neural network modelling. Speech recognition techniques have historically included statistical method modelling, such as with using hidden Markov models, and have only been surpassed in the last decade with the, more robust, neural network models. This paper focuses on creating a Croatian speech recognition system, as well as with methods that are used in audio signal processing in the first four chapters. The last two chapters focus on the experiment, in which the speech recognition system using HTK and Julius was created, and also the development of a LSTM language model using Keras. FastText was also used for word embeddings. The conclusion of this paper is that replacing Hidden Markov models with neural networks is not a simple issue, and it is necessary to plan accordingly when thinking about how to employ such a language model sucessfully, considering the incompatibilities between both architectures.