European freight railway operators, aiming to remain competitive in the Single European Railway Area, cannot enhance their competitiveness relative to road transport or meet user demands without investing in new technologies. The efficiency brought by these new technologies is a current and relevant topic (state of the art), but it must be verifiable and measurable. This is particularly important for railway operators on small networks, such as Republika Srpska Railways JSC. (Serbian: Zeljeznice Republike Srpske A.D. - ZRS), which have limited resources for investment. Therefore, it is crucial to identify and implement new technologies that will yield the greatest effect, thereby improving the competitiveness of the operators in the open market. The primary goal of this paper is to quantify the effects of introducing specific new technologies on the competitiveness of ZRS. The competitiveness of freight rail services is evaluated from the perspective of users (end users, freight forwarders, and logistics companies). By developing a set of KPIs for ZRS as a freight railway operator, it is possible to measure the direct impact of new technologies and technical solutions on the competitiveness of ZRS's rail services, thereby addressing identified weaknesses.