2008 Russian military reform: An adequate response to global threats and challenges of the twenty-first century?*

La reforma militar rusa de 2008: ¿una respuesta adecuada a las amenazas y retos del siglo veintiuno?

La réforme militaire russe de 2008: une réponse adéquate aux menaces et aux défis du XXIe siècle?

A reforma militar russa de 2008: uma resposta adequada às ameaças e desafios do século XXI?

Andres Eduardo Fernandez-Osorio ^a

*University College London & National Research University - Higher School of Economics. Research paper based on the third and fourth chapter of the author's master dissertation, completed at University College London, United Kingdom and National Research University - Higher School of Economics, Russian Federation.

^aMajor in the National Army of Colombia. MA in Economy, State and Society: Politics and Security -University College London, United Kingdom. MA in International Relations in Eurasia - National Research University Higher School of Economics, Russian Federation. BA in International Relations and Political Studies - Universidad Militar Nueva Granada, Colombia. BSc in Military Sciences - Escuela Militar de Cadetes General Jose Maria Cordova, Colombia. Contact: foandres@live.com.

Recibido: 16 de noviembre de 2015 • Aceptado: 10 de diciembre de 2015

Abstract

Applying a military sciences' methodology namely DOTMLPF, this paper examines the 2008 Russian military reform to determine whether or not it will prepare Russia to face contemporary threats and challenges. Analysing published studies with the results of a DOTMLPF assessment and essentials of modern military thinking, this paper offers conclusions on the effectiveness of the reform. Besides, it provides a new perspective when analysing military reforms and emphasises the significance of Russia and its military capability for the global balance of power.

Keywords: DOTMLPF, military doctrine, modernisation, Russian military reform.

Resumen

Mediante la aplicación de una metodología de las ciencias militares llamada DOTMLPF (por sus siglas en inglés), este artículo examina la reforma militar rusa de 2008 para determinar si preparará a Rusia para afrontar amenazas y desafíos contemporáneos. Al analizar la literatura existente con los resultados del análisis DOTMLPF y los principios de pensamiento militar moderno, este artículo ofrece conclusiones sobre la eficacia de la reforma. De igual forma, provee una nueva perspectiva cuando se analizan reformas militares y enfatiza la relevancia de Rusia y su capacidad militar para el balance de poder global.

Palabras clave: DOTMLPF, doctrina militar, modernización, reforma militar rusa.

Résumé

L'application d'une méthodologie science militaire appelée DOTMLPF, cet article examine la réforme militaire russe de 2008 afin de déterminer si la Russie se préparent à faire face aux menaces et défis contemporains. En reliant la littérature existante avec l'analyse résulte DOIMLEPI et au début de la pensée militaire moderne, cet article fournit des conclusions sur l'efficacité de la réforme. De même, il fournit une nouvelle perspective lors de l'analyse des réformes militaires et souligne l'importance de la Russie et sa capacité militaire à l'équilibre du pouvoir mondial.

Mots-clés: DOTMLPF, doctrine militaire, modernisation, réforme militaire russe.

Resumo

Aplicando uma metodologia chamada DOTMLPF ciência militar, este artigo analisa a reforma militar russa de 2008, para determinar se a Rússia se preparam para enfrentar as ameaças e os desafios contemporâneos. Ao vincular a literatura existente com a análise resulta DOTMLPF e pensamento militar moderna, este artigo fornece conclusões sobre a eficácia da reforma. Da mesma forma, ele fornece uma nova perspectiva ao analisar as reformas militares e enfatiza a importância da Rússia e sua capacidade militar para o equilíbrio de poder global.

Palavras-chave: DOTMLPF, doutrina militar, modernização, reforma militar russa.

In Russia, you need to enact reforms quickly; otherwise they mostly do not work out and are inhibited.
Witte (1905 as cited in Barabanov, Makienko and Pukhov, 2012, p. 3)¹

Introduction

Since it was announced in September 2008, Russia's new military reform has received an increasing amount of critical attention from the international community; after all, military capability has a pivotal role on policy making and its rational usage is one of the greatest challenges in the modern world.

The reform, planned to run beyond 2020, aims to convert Russia's military into a modern and deployable multi task force prepared to manage global threats and challenges of the twenty-first century by solving fundamental problems such as ageing equipment, corruption, and limited readiness. In addition, it projects a structural transformation to bring the Russian military closer to international standards, reducing the force structure and maximising its combat readiness, re-equipping the armed forces with modern gear and defence systems, improving military doctrine and instruction programs, training new officers and non-commissioned officers with an enhanced network of military schools, and solving salary and wellbeing problems (Barabanov, Makienko and Pukhov, 2012, p. 6; Fernandez-Osorio, 2015, p. 74).

Despite a relatively little early international interest on this reform due to the ineffective results of past Russian military restructurings, recent events such as Russia's intervention in Crimea and the beginning of a military campaign in Syria have heightened the need for cautious assessments on its concrete results.

A much debated question on the reform is the appropriateness of its methods and approaches to effectively modernise Russia's military, and hence, whether or not it will prepare Russia to face contemporary global threats and challenges. The generalisability of much published research on this issue is problematic due to the usage of methodology more associated with traditional disciplines when analysing the reform, showing an overwhelming pessimism about its effectiveness among Russian and Western commentators alike.

This paper seeks to cast light on these difficulties by analysing the 2008 reform with a military sciences' methodology used in modern forces such as NATO (North Atlantic Treaty Organization) and the OSCE (Organization for Security and Co-operation in Europe), namely a DOTMLPF (Doctrine, Organisation, Training, Materiel, Leadership, Personnel, and Facilities) assessment. Linking the existent literature with the results of a DOTMLPF assessment and essentials of modern military thinking, this paper aims to elucidate the extent to which the 2008 reform will be effective preparing the Russian military to deal modern defence and security scenarios.

The debate on the 2008 military reform: diverse approaches, common conclusions

An intense debate on the 2008 military reform (henceforth MR) has been growing in recent years, based on studies of well-known experts and institutes that have provided various insights and perspectives on the theme. Nonetheless, the numerous studies completed have tended to use methodology more frequently used to analyse civilian macro-projects and as a result, there is a consensus among researchers regarding the failure of the reform giving five main explanations: budget limitations, systemic corruption, opposition, technological lag, and demographic constraints. In addition, there is little evaluation of 2008 MR achievements so far, insufficient suggestions of other reasons for the slow pace of the reform, and few assessments on the preparedness for modern threats and challenges that this reform will offer to Russia.

Budget limitations

Supporters of the failure of 2008 MR due to budget limitations argue that military forces' development is highly dependable on the adequate financial support from the state and the correct distribution of these assets according to priority plans that will lead to modernisation. As suggested by the Rand Corporation(2000 as cited in Hedenskog and Vendil Pallin, 2013, p. 103), a "country's defence budget is the most general single measure of the resources provided to the military as it conveys a sense of the size of the military establishment and the relative importance of defence in comparison with other public spending". Consequently, the analysis of Russia's economy and its defence budget for the current and forthcoming years may lead to conclusions on the growth potential of its military forces (Figure 1).

Russia's Military Expenditure (ME)² (Table 1) / National Defence Expenditure (NDE) (Table 2) has increased from an average of 2.5 per cent of gross domestic product (GDP) in 2008 to approximately 4.7 per cent in 2014 - principally due to the precedence given to armament modernisation through the State Armaments Program for 2020 (GPV-2020)³ and personnel restructuring (Figures 2 and 3). Nevertheless, for some experts (Renz, 2010, p. 60; Cooper, 2012, p. 12,15; Hedenskog and Vendil Pallin, 2013, p. 117), the 2008 MR was based on a positive appraisal of the Russian economy and projections of defence budget allocations were unrealistic in regard with Russia's economic limitations.

Similarly, future allocations for 2016 (3.9 per cent of GDP) are highly unlikely to be achieved due to current stagnation trends of Russia's economy and its forecasted deceleration (Figure 4). As argued by the World Bank (2014, p. 1) "the lack of more comprehensive structural reforms has led to the erosion in businesses' and consumers' confidence, (...) became the decisive factor for the downward growth projections for Russia".

One notable weakness of this approach is the concept that Russia's defence budget has increased exaggeratedly over the past years, going against current world tendencies (RT, 2014; DW, 2014; Norton-Taylor, 2014). Nevertheless, Russia's military has not being equipped with modern weaponry since the breakdown of the USSR and a significant part of its inventories are inoperative or obsolete. "In 1992-2008, there were no significant arms acquisitions for the general-purpose forces [and] according to some estimates, 60-70% of the Russian tanks and armoured vehicles broke down over the course of the South Ossetian conflict in 2008" (Makarychev and Sergunin, 2013, p. 357). Moreover, as reported by The Military Balance, for 2010 "about fourteen percent of the tanks and about twenty percent of the artillery pieces were in active units" (Vendil Pallin, 2012, p. 105).

When analysing this perception in light of the ratio between the size of the military forces and the National Defence Expenditure of the top five countries with the highest defence expenditure per year, it is possible to point out a discrepancy. On the one hand, when defence expenditure is measured in USD billion (Table 3), Russia appears to be a high military spender in 2012 and 2013, however, when defence expenditure is measured as a percentage of GDP (Table 4), other countries surpass Russia by far. For instance, Russia's defence expenditure for 2012 (2.9% GDP) and for 2013 (3.1% GDP) falls behind expenditure for the same years by Afghanistan (10.5 % and 13.8% GDP), Saudi Arabia (7.9% and 8.0% GDP) and Israel (7.8% and 7.2% GDP).

On the other hand, the ratio between defence expenditure and military force size provides a plausible indication of its development possibilities. Ideally, the bigger the military force, the bigger should be the budget to maintain it, otherwise the probability of falling behind the capabilities of adversaries is highly likely. "The planned military spending formula can be defined as the maximum possible spending, taking into account the economic potential of Russia and the minimum necessary to reconstruct an effective military machine" (Barabanov, Makienko and Pukhov, 2012, p. 14).

In the case of Russia, the obligations acquired from the former Soviet military -estimated by 1985-1986 at about 4.9 million active servicemen (Nichol, 2011, p. 1) - in terms of pensions and welfare benefits, plus operation costs of the actual Russian military pose a huge challenge to the 2008 MR. As suggested by Arbatov (2008 as cited in Liaropoulos, 2008, p. 47), "Russia's military spending is comparable with those of Great Britain or France, whose military forces number 250,000 men". To illustrate this notion, Table 5 depicts the ratio between defence expenditure and military personnel only. In the case of Russia, the ratio is 81 USD per service member per year, however, if the MoD's official figure of 1,000,000 service members is used, this ratio falls to 68.2, a figure that may be very low to cover military forces of the complexity as Russia's.

These financial constraints are one of the reasons why Russia opted to modernise its military, looking forward to guaranteeing its future financial sustainability. Nonetheless, the deceleration of Russia's economy for the forthcoming years, confirmed by the findings of an International Monetary Fund inspection in Russia that forecasted for 2014 a decrease in Russia's GDP from 1.3 to 0.2 per cent and an increase in the inflation rate of over 6 per cent (Interfax, 2014), on top of the economic sanctions against Russia because of the crisis in Crimea, may confirm the failure of the 2008 MR by budget limitations.

Systemic corruption

Other authors attribute the failure of the 2008 MR to systemic corruption by arguing that dishonesty in Russia is still rampant and seems to have shown no signs of major improvement (Rotberg, 2009; Cheloukhine and Haberfeld, 2011; Vendil Pallin, 2012; Orttung and Latta, 2013; Cimbala, 2013); in fact, public perception in Russia towards achievements in this field is relatively low (Table 6). Moreover, for some experts, corruption was one the main reasons for the appointment as Minister of Defence of a civil servant, Anatoly Serdyukov, with a low-level military background - just compulsory military service in the Soviet Army in 1984 - (Herspring, 2008, p. 20; Bryce-Rogers, 2013, p. 346), i.e. to eradicate, or at least reduce, this malpractice.

Nonetheless, Russia's corruption perception index for 2013, prepared by Transparency International (2013; RIA Novosti, 2013e), was reported as twenty-eight points out of a possible one hundred, with zero being a highly corrupt country and one hundred, a highly honest one (Figure 5). Predictably, for 2014 Russia received a similar score, twenty-seven points. Similarly, a 2012 research on corruption in the defence sector undertaken by the National Research University Higher School of Economics placed the Russian defence establishment in the lowest ranks (the poorest evaluations) in regard to the risk of corruption and the quality of anticorruption policies. Hence, publicised achievements on eradicating corruption are likely to be elusive.

Recent information indicates that corruption levels remain the same or are even worse than in previous years, where "accounts for 70 percent of the budgetary resources [were] used for purposes other than those officially confirmed" (Wegren and Herspring, 2009, p. 269). In particular, the Office of the Russian Prosecutor General announced in July 2013 that "the financial cost of corruption uncovered in the Russian Armed Forces this year has soared 450 percent from last year to over 4.4 billion roubles ($130 million)" (Miller, 2013, p. 1). Given the above points, besides the prosecution of 144 high-ranking MoD officers (Chaykovskaya, 2012, p. 1), it may be possible to conclude that 2008 MR methods - focused on shifting financial management from military personnel to civilians - has not provided the expected outcome. On the contrary, as reported by the Prosecutor General's Office "the policy of ridding the Armed Forces of ancillary functions and transferring them to outside organizations, or outsourcing, has caused hundreds of millions of roubles in damages" (RIA Novosti, 2013a). Moreover, as now "every third corruption offense is committed by civil servants and civilian personnel" (RIA Novosti, 2013a), it seems that corruption just changed from military to civilian hands, while maintaining the same intensity. Nonetheless, despite its widespread prevalence, it is highly unlikely that corruption alone led to the failure of the 2008 MR. It may undermine and delay some important projects, but disrupting the modernisation of the Russian military and its enormity is improbable.

Opposition

Followers of the theory of a 2008 MR collapse due to opposition maintain that the implementation of the reform failed as a negative image of it was projected, thus creating a great deal of opposition within military and political circles, influential enough to make the reform fail (Wegren and Herspring, 2009; Thornton, 2011; Lannon, 2011; Nichol, 2011; Bryce-Rogers, 2013). Opponents' perception is that reform goals and methods aim to reduce the military at all costs without taking into account the security and defence challenges of Russia and thus, "are destroying Russia's military capability" (Eckel, 2008).

As argued by Wegren and Herspring (2009, p. 271) "conflict is an inevitable part of change in any bureaucracy, but it is especially prominent when one is dealing with a tightly knit, conservative group, especially one like the Russian military. By definition, it opposes the introduction of civilian outsiders". Former military officers dismissed during Serdyukov's tenure led an important part of this opposition, as, according to their vision, only the military has the understanding, experience, and vision to make decisions on military grounds since civilians do not understand the concept of defence/security and cannot give a clear verdict on these matters.

One of the most publicised criticisms came from the conflictive relations between the Chief of the General Staff of the Russian Armed Forces, General Yury N. Baluyevsky -the highest position for an active Officer in the Russian military and main advisor of the Minister of Defence - and Defence Minister Serdyukov. General Baluyevsky opposed the appointment of more than 30 civilian specialists as military advisors, who were responsible for military-strategic issues, along with financial and economic matters. "Apparently the ambitious incompetence of some of these advisors, who are for some reason a-priori supported by the new minister, provoked Baluyevsky's decisive protest, which he expressed in the form of his resignation" (Wegren and Herspring, 2009, p. 274).

As logical as this vision may sound, it is not strictly necessary - although desirable - to have a background in defence/security fields to manage military forces as a Minister of Defence. To this extent, Minister Serdyukov proved his value in leading the MoD for almost six years (February 2007-November 2012) while reaching some targets on the armaments procurement system, salary increases and housing solutions as well as selling off obsolete military weaponry (Litovkin, 2011).

Every social group with distinctive characteristics - such as the military - may reject outsiders when their autonomy feels threatened; in the same way, the Russian military, accustomed to years of having an important role in the society, felt discomfort when a civilian was appointed as Minister of Defence. Furthermore, who can make the best decisions? Is the answer the economist/politician with a long record of effectiveness but poor military knowledge, or the military person with extensive experience but overshadowed by years of malpractice? It is precisely the existence of such dichotomies that provides the richness to democratic discussions on military matters and its openness to society's accountability.

Nevertheless, the criticism originated not only in the military but from a wide range of political opposition as well. On the one hand, the communists and nationalists concurred with the arguments of the military by accusing the government and the Minister of Defence of "destroying the armed forces by the ill-advised and chaotic reform that is based on the business-like or commercial approach" (Sergunin, 2012, p. 254; Makarychev and Sergunin, 2013, p. 361). Besides, Serdyukov's appointment was perceived both as a reward, for his key role as the head of the Federal Tax Service of Russia in prosecuting the Kremlin's political opponents and especially in terminating Mikhail Khodorkovsky's financial empire (McDermott, 2012), and as a political favour since Serdyukov is the son-in-law of Viktor Zubkov, a close ally of Vladimir Putin (Kommersant, 2007).

On the other one hand, the liberals were not satisfied with the openness of the MoD to civilian participation as, despite the appointment of several civilians as high-level advisors, the majority of the staff remained with the military. Besides, notwithstanding that the reform has assigned to civilian bodies contracting and monetary responsibilities, the procurement agencies are independent of, but still subordinate to, the MoD (Cooper, 2012, p. 10). Additionally, the contribution of civilian industry to the supply of new armaments is still irrelevant, as the majority of enterprises are state owned or controlled by the state (Cooper, 2012, p. 21).

The ousting of Minister Serdyukov was seen as a victory of opposition. On the one hand, it was considered a revenge of the old military establishment and leading arms makers who "have conspired to bring Serdyukov down" (Isachenkov, 2012). As suggested by Mankoff (2012) "Serdyukov's refusal to buy sub-standard arms from the Russian defence industry represented a threat to the survival of one of Putin's key constituencies", hence his dismissal was just a matter of time. On the other hand, Serdyukov's dismissal was considered a consequence of his conflicts with key players in Putin's inner circle such as the presidential chief of staff, Sergei Ivanov and Deputy Prime Minister Dmitry Rogozin, who exploited military unease with the reforms to involve Serdyukov in the Oboronservis investigation (Sukhov, 2013).

Opponents may pose a threat to the modernisation of the Russian military due to its interference in almost all levels of command and the execution of orders; however, it seems, to date, that opposition is not strong enough to signify the failure of the 2008 MR. Moscow is committed to the modernisation of the military as it will enforce its policies locally and internationally; hence, it is highly likely that Russia's leaders will take any necessary measure to remove opponents of their interests and strategies.

Technological lag

Some scholars blame the inability of the Russian Military-Industrial Complex (MIC) to pro-vide modern armaments and defence/security systems as the reason for the failure of the 2008 MR (Herspring and McDermott, 2010; McDermott, 2011a; Blank, 2011; Sergunin, 2012; Makarychev and Sergunin, 2013). Indeed, there is a large volume of published studies describing the lag of Russian science and technology in comparison to the West, and its implications for defence research. Similarly, the inability of the Russian MIC to both design new armaments - not just a modernisation of old Soviet-era models - and to offer rational prices for acquisition, maintenance and spare parts has been widely criticised by experts and by the Russian MoD.

For instance, the exaggerated price and conditions demanded by the MIC for amphibious assault ships forced the Russian MoD to look for options in the international market. As a result, a 1.12 billion EUR contract was signed in 2011 with France to buy four French BPC-210 Mistral-class amphibious assault ships, including the transfer of technology and joint production for future ships (RIA Novosti, 2011). A similar story happened with the Searcher-II Unmanned Aerial Vehicles (UAV) that had to be bought from Israel in a 400 million USD contract in 2010 because of the limitations of Russian-made drones used in 2008 during the war against Georgia (Fater, 2010; RIA Novosti, 2012b). As argued by Bryce-Rogers (2013, p. 362) "despite the fact that Russia is the world's second largest supplier, accounting for 23 percent of arms exports globally, the Russian defence industry remains generally incapable of producing 21^st century equipment. Rather, much of the latest equipment is based on outdated designs that were created ten or fifteen years ago".

Notwithstanding the necessity of procurement of new weaponry and the upgrade of old inventories, the 2008 MR and the Russian MoD have had an ambiguous policy. On the one hand, a major emphasis has been put on the self-procurement of weapons and developing its own systems, based on Russian science and technology, by "improving the qualitative level of the defence industry complex" (Russian Federation Ministry of Defence, 2010, pp. 15,21), thereby avoiding dependence on other states and theirs foreign policy. On the other hand, the MIC has been encouraged to increase the quality of its modernisation plans for Soviet-era platforms and new designs through cooperation with foreign industry (Table 7). This has included a contract signed in 2012 with Italy for 60 M65E Lynx Light Multirole Armoured Vehicles (LMAV), manufactured by Iveco Defense Vehicles (RIA Novosti, 2012d) and Western electro-optic components to modernise T-90 main battle tanks (Sergunin, 2012, p. 256).

The technological modernisation required by the military has pushed MIC capabilities to its limits. Indeed, in 2006, Vladimir Putin announced a 189 billion USD GPV re-equipment programme for 2007-2015 but its outputs were unsatisfactory, as several orders were not completed due the lack of capacity of the industry. Hence, a new GPV for 2010-2020 was prepared both to solve failures in the former and to equip the military with a proportion of 30 per cent of modern arms by 2015 and at least 70 per cent by 2020 (Bryce-Rogers, 2013, p. 361; Hedenskog and Vendil Pallin, 2013, p. 121). However, from this new 19 trillion RUR budget, only around 10 per cent has been assigned for research and development, 10 per cent for repair and upgrading of older platforms and equipment, and 80 per cent for new weapons procurement (Hedenskog and Vendil Pallin, 2013, pp. 111-112). Besides, this new GPV-2020 decreased the share of state financial support for the MIC. GPV-2015 funded 80 per cent of project costs while the MIC funded the 20 per cent remaining, however; GPV-2020 funds only 60 per cent and the MIC should fund 40 per cent.

The points given above have posed both a threat and a challenge to the Russian MIC. On the one hand, there is a threat to MIC's sustainable commercial plans, due to the negative results of several studies (Adamsky, 2008; Kosals and Izyumov, 2011; Wengle, 2012; Hedenskog and Vendil Pallin, 2013; Roffey, 2013; Oxenstierna and Westerlund, 2013), which show the difficult situation the MIC is confronting during the present years as the main percentage of the GPV-2020 goes to the acquisition of new armaments. Ageing tools and machinery, high-skilled workers reaching their retirement age and an incapacity to attract young talented scientists, weak research and development corps, poor innovation, in spite of the usage of proven Soviet-era platforms, and an old-fashioned management system, are some of the problems that will make the situation even worse, if they are not solved. On the other hand, the MIC is faced with a challenge as it is obliged to promptly create strategies that will modernise its companies and its products, being competitive in quality and price in the international market, and being profitable, while complying with Russia's requirements, otherwise the Russian MoD may shift its procurement to the international market.

With the recent events in Crimea and Syria, the European Union and the United States will probably restrict or deny any further sale of weapons or military technology to Russia; hence, Russia's military will depend solely on the MIC. If the MIC fails to equip the Russian military adequately, the balance of power and Russia's strategic advantage will be affected negatively and therefore, it may signify the failure of the 2008 MR.

Demographic constraints

Finally, other experts (Nichol, 2011; de Hass, 2011; Barabanov, Makienko and Pukhov, 2012; Blank, 2012; Hedenskog and Vendil Pallin, 2013) support the view that the failure of 2008 MR is due to demographic constraints. They argue that despite the reduction in the number of conscripts per annual draft quota - approximately 444,000 in 2011 - (Nichol, 2011, p. 36) and the decrease in service time - from 24 months to 18 months in 2006, and then to 12 months in 2008 - the diminution in births after the breakdown of the Soviet Union and the poor health conditions of young Russian men make it rather difficult to achieve 2008 MR expected goals in conscription and professionalization (Kamenev, 2009). Draft boards have been forced to include personnel below medical standards or even men with criminal, or drug-abuse records in order to comply with draft quotas, decreasing profoundly the quality of the conscripts and the readiness of the military (Nichol, 2011, p. 36).

Similarly, it has been necessary to draft personnel from challenging areas such as the North Caucasus, which historically has led to ethnic clashes and thus increased the harassment (dedovshchina) and crime in barracks (Barabanov, Makienko and Pukhov, 2012, p. 13). Paradoxically, experts claim that conscripts from North Caucasus are the most physically adapted to, and more motivated for, military service, but the ethnical differences may affect the loyalty of these soldiers if they are employed in the North Caucasus or abroad (Nichol, 2011, p. 37).

Given the above points on conscription, the 2008 MR goals on professionalization through contract soldiers (kontraktniki) and the recently created Non Commissioned Officers (NCO) corps have not been met. The original strategy aimed to reduce conscription by moving towards a professional corps of contract soldiers, supervised by NCOs and led by officers - resembling Western military forces - however, the number of contract soldiers and NCOs has not met the required levels while conscription has decreased year-on-year (Gresh, 2011, p. 216). Consequently, real combat readiness has been jeopardised; for instance, as of November 2012, the military was undermanned by about 40 or 60 per cent (RT, 2012b; RIA Novosti, 2013b) of the required staff for combat operations. Even though some administrative measures have been authorised such as banning draft dodgers from state service (RT, 2012a), more effective decisions still need to be taken if the Russian MoD wants to achieve its goals on time.

The errors in demographical constraint projections made by the MoD pose one of the imminent risks of failure to the 2008 MR. If no adequate measures are taken to consolidate a professional military, non-dependent on recruitment, it is highly improbable that the draft would fulfil the expected personnel quotas. Similarly, if no improvement in the conditions of professional military personnel are made, significant numbers of them will continue abandoning their duties or quitting after the end of their first contract.

Limitations on the debate

One major drawback of the above approaches is that experts have cited other factors that may also have undermined the effectiveness of the 2008 MR. Perhaps one of the most serious of these undervalued factors is the lack of adequate leadership, in terms of the authoritative management style of high-level officials, including former Minister of Defence Anatoly E. Serdyukov. Various reports over the past years have mentioned Serdyukov's difficult relations with high-ranking military officers - most probably originating from prejudices regarding corruption and malpractice - being a barrier to proper teamwork and coordination, and therefore, a setback for modernisation (Eckel, 2008; Felgenhauer, 2008, 2009; Gorenburg, 2009; Flintoff, 2012; Kalinin, 2012; Litovkin, 2012; Jensen, 2013; RAPSI, 2013; Golts, 2013). Nonetheless, the MoD discretely dismissed those reports by presenting them as part of a profound reorganisation to get the military forces back on track.

Nevertheless, recent evidence (The Economist, 2012; Weitz, 2012; Cohen, 2012; Gorenburg, 2013b; Kramer, 2013; Sukhov, 2013; RIA Novosti, 2013g) and Serdyukov's dismissal in November 2012 (Rosenberg, 2012; Isachenkov, 2012; Kramer, 2012; Krainova and Oliphant, 2012; Andreyeva, 2012), suggest the reports on Serdyukov's leadership malpractices were correct and they signified an augmentation of transition problems to a new military due to a regular rejection of experts' advice.

Serdyukov's refusal to recognise and use the expertise of military staff and the lessons of Russia at war, led to regular clashes with high-ranking officers, lack of necessary team work, and disrespectful behaviour among the personnel (Eckel, 2008; Andreyeva, 2012; Krainova and Oliphant, 2012; Kramer, 2012). Reciprocal mistreatment at the MoD was regularly reported (Bryce-Rogers, 2013, p. 346); for instance, Minister Serdyukov privately referred to top officers as little green men - referring to the colour of uniforms used by the army - while high-ranking officers referred to the minister as general chair or furniture boss - referring to Serdyukov's former job in the furniture market - (The Economist, 2012, p. 1; Nemtsova, 2012, p. 1; McDermott, 2012, p. 2).

Serdyukov's methods of rejecting criticism and purging opposition were widely commented on by experts (Soloviov, 2008; Pukhov, 2008; Litovkin, 2010; Gorenburg, 2013b; Bruneau and Matei, 2013, p. 113) and even by the Russian President, Vladimir V. Putin, who, in 2013, stated that "not all the decisions previously made on Russian military reform had proved to be useful, but constant shifts in the general course of the reform program were unacceptable" (RIA Novosti, 2013c).

This problem of opposition inside the MoD may provide hints on the lack of adequate civil-military relations as it is difficult to believe that every officer in the MoD was corrupt or completely negligent. As Wegren and Herspring (2009, p. 282) commented on General Baluyevsky's case "why not incorporate [Baluyevsky's] plan into Serdyukov's austerity measures? Doesn't the defence minister understand that the two should go together?" It is difficult to provide a single answer, but it may be linked to the weak image of Serdyukov in his military and political opponents' mind due to his civilian background and thus, his inability to act as the necessary reformer to bring the military back on course. It may also be due in part to the limited seriousness assigned to the reform as it was perceived as a recreation of past failed reforms and not as a clear plan to modernise the military (Clover, 2009). If Russia wants to achieve a real modernisation, it should be understood that the military represents both the state and the society; hence, enhanced participation mechanisms, broad and sincere critique possibilities, and increased openness to accountability should be introduced, with the participation of the military as another actor in the discussion. Being part of the design of the reform will commit the military to the goals to be achieved and will assist an earlier transition to more democratic practices.

Another of the most serious undervalued factors - and related to the lack of adequate leadership - is the absence of proper judgment and planning regarding the needs of the Russian Military. Similarly, several reports since 2008 have identified many erratic or incorrect decisions of the MoD as an obstacle to modernisation. These measures, many of them presented as necessary to counteract corruption - such as the creation of Oboronservis and the reduction of high-education military institutes - have been proven to be erroneous or have been reversed by the new Minister of Defence Shoigu (McDermott, 2013; RIA Novosti, 2013c; Weir, 2013; Felgenhauer, 2013a; Rusmilitary, 2013; Felgenhauer, 2013b; Gorenburg, 2013c). However, the reasons the Kremlin had to appoint Mr Serdyukov as Defence Minister in February 2007 were related to his achievements as head of the Federal Tax Service of Russia and the necessity to undertake a profound modernisation in the military bearing the political cost, which that decision could signify (Kommersant, 2007; Saradzhyan, 2008). As defined by Bartles (2011, p. 80), Serdyukov's approach corresponded more to that of a businessman than to a Minister of Defence.

The dissimilar set of goals contained in the 2008 MR including: inadequate methods to bring them into practice - such as promoting full professionalisation of the forces but supporting troops' main corps with compulsory draft - and the disconnection between the assessment of threats and challenges facing Russia, the reform goals and the capabilities planned for the military, all had a devastating effect on the credibility of both the reform and Minister Serdyukov, which then became political costs impossible to bear, despite the initial "carte blanche political support from the ruling duumvirate of Dmitry Medvedev and Vladimir Putin" (McDermott, 2011b, p. 3).

Beyond all its deficiencies, it is necessary to acknowledge Russia's necessity for a military reform, to both leave behind Soviet-era structures and policies and to consolidate Russia's democratic processes. Nonetheless, only a comprehensive well-designed strategy will effectively modernise the Russian Military, providing an adequate answer to the global threats and challenges of the twenty-first century.

Discussion: Analysing the 2008 Russian Military Reform (a DOTMLPF assessment)

DOTMLPF methodology analyses the doctrine, organisation, training, materiel, leadership, personnel, and facilities involved in a military project, aiming to suggest an ideal configuration to meet the national security and defence policy, the military doctrine, and to provide an estimation of Russia's Military Capabilities (RMC), affected negatively by the lack of DOTMLPF variables.

The central focus of a DOTMLPF assessment is to identify capability deficits in a military project, providing an estimation of the reliability of its scope and methods, through a functional mission analysis and a functional needs analysis. The outcome of these analyses is a decision-making tool in the form of an ideal force structure, in four areas required in a modern military: its basic requirements, its enhanced characteristics, the focus of its capacities, and the scenarios of its operation. In other words, the minimal requirements that Russia's military has to have to comply with the 2008 MR, the 2010 Military Doctrine and the MoD objectives (Figure 6) as well as improving Russia's Military Capabilities (RMC).

Figure 7 serves to highlight the absolute importance of both an adequate leadership, and proper judgment and planning to comply with Russia's military projections. The success of every element and of the whole projection depends exclusively on correct management and appropriate decision-making. Similarly, Figure 7 will serve as a basis to identify the elements most affected by the lack of variables in the DOTMLPF assessment, depicting how the malfunction of any of these will affect the whole projection.

Doctrine

An essential part of a modern military force, if not the most important, is doctrine and its coherence with upper-echelon documents within the legal framework of a state such as international law and treaties, the constitution, domestic laws, national security strategy and the foreign policy concept. This assures a proper defence/military decision-making, the legitimacy of the methods used by the military and better democratic control over the forces and the use of its resources. Besides, it guarantees that the military is organised, trained, equipped, led, manned, and supervised appropriately, according to state policy and goals.

The Russian Federation has produced several military doctrines since the breakdown of the Soviet Union, including the version of May 1992, which was only in draft form due to its rejection by the Supreme Soviet of the Russian SFSR (Soviet Federative Socialist Republic); the version of November 1993; the version of January 2000; and finally, the current version of February 2010. Along with these two National Security Concepts have been published, in December 1997 and October 1999; and a National Security Strategy to 2020 approved in 2009, which has extended the notional structure given.

Russia's latest military doctrine (released in 2010) was designed as the theoretical support for the 2008 MR and is defined as "the fundamental strategic planning documents in the Russian Federation and constitutes a system of views officially adopted in the state on preparations for armed defence and on the armed protection of the Russian Federation" (Russian Federation Ministry of Defence, 2010, p. 1). However, several discrepancies in this document can be pointed out, implying an absence of both adequate leadership, and proper judgment and planning for the needs of a doctrine of the Russian military.

Firstly, a contradiction, related to the fact that the reform and a significant part of the transformation achieved with it, was enforced almost two years before the issue of the actual doctrine, in 2010. Consequently, the scope and methods of the 2008 MR were drawn from past policies, non-concordant with contemporary Russia, and designed without having proper knowledge of modern Russia's goals and objectives for the military. The result of shaping the 2010 military doctrine to the results of the 2008 MR and not the reverse, as it should be, has signified contradictory plans such as the precise number of Army Brigades required, and thus, the equipment, manning and training necessary. While the 2008 MR established the necessity of creating only 85 combat-ready brigades, in 2010, due to the multiple references in the military doctrine to the necessity of the mobilisation system, the high-ranking military officers requested the creation of 180 brigades, including their mobilisation capabilities (McDermott, 2011b, p. 24).

Secondly, an ambiguous outcome, related to the fact that a Security Council commission chaired by General Yury N. Baluyevsky (deputy secretary of the council and former Chief of the General Staff), a well-known opponent of the 2008 reform, was responsible for developing the 2010 military doctrine (Golts, 2008). Added to this, the conservative advisors to the Security Council of the Academy of Military Science and the General Staff of the MoD, who insisted on the necessity of Russia's capability to fight a world war, created the threatening and dangerous perception of the 2010 Military Doctrine, and thus, some of its content reflects the views of a superpower with a Soviet-era outlook.

Thirdly, there is an uncertainty related to the absence of trustworthy studies and statistics on defence/security grounds in the Russian military, and therefore, for the design of the 2010 Military Doctrine and the 2008 MR and its methods. As argued by (McDermott, 2011b, pp. 11-12) the excessive secrecy enforced during Stalin's period led to the removal of military statistics from the curricula of military academies; besides, statistical results were hidden during the era of the Soviet Union. Consequently, Russia's military inherited a lack of an adequate source of military statistical information, including performance and administration indicators usable to form an appropriate judgment and for planning the requirements of the military. This had undermined the decision-making process and Russia's own-capability assessments in relation to creating the best military force possible.

Finally, the contradictions in the 2010 Military Doctrine have impeded military strategy and tactics to be modernised. For instance, new tactics for ABs, and the concept of joint operations for USCs, have not been produced and the operational art still conserves ideas of the Soviet military (Hedenskog and Vendil Pallin, 2013, p. 80). Similarly, no advances other than the use of force, has been made in strategies for handling unstable regions such as the North Caucasus, or for post-conflict scenarios avoiding the emergence of new conflicts. In addition, sources for military thinking have been created only recently; the Military Scientific Council was created in April 2011 and the Advanced Research Foundation was created in October 2012; but the main foundation for improvement remains with the conservative structures of the General Staff and the Academy of Military Science.

In light of the above, it is possible to conclude that in the doctrinal area the 2008 MR has failed to prepare Russia's military appropriately to manage the global threats and challenges of the twenty-first century. Figure 8 highlights the elements of RMC that have not been achieved due to the lack of an adequate doctrine for its forces. It confirms the view that the absence of an adequate doctrine has created uncertainly in the military in respect of what organisational decisions to take and hence the basic requirements (organisation, equipment, manning, training, mobility and interoperability) have been undermined. Besides, enhanced capabilities, as an outcome of the basic requirements, have been damaged. Consequently, the capacity to deal with diverse circumstances in different scenarios has been reduced. Despite the undeniable progress achieved in reference with the Soviet military, there are still numerous concepts, strategies, and theories to be researched and developed in order to reduce the necessity for the use of force, while maximising its effectiveness.

Organisation

The 2008 MR enforced a radical organisational transformation of the military. The C3 structure was modified by merging the MD/USC thereby reducing their number from six to four, creating operational responsibilities of the majority of troops and assets in their jurisdiction.

Similarly, changes were made in each service (Army, Navy, Air Force) and arm (Strategic Missile Forces, Aerospace Defence Forces, Airborne Forces). However, apart from the creation of Air Bases as the new basic unit of the air force to improve the administration of air assets across the MD/USC, and the disbandment of some units of the Navy and Marines, the biggest strategic change has been the disbandment of Army Divisions in favour of Army Brigades as the new basic unit of the Army to improve its mobility (Barabanov, Makienko and Pukhov, 2012, p. 22).

The adoption of the Brigade structure, used in Western militaries, has been both praised and criticised. On the one hand, Divisions are territorially based complex structures organised and equipped to confront large conventional wars but with the capacity to face almost any type of aggression due to its possession of organic combat support. In other words, a Division has its own artillery, army aviation, air defence, tanks, engineers, etc. that grant it independence of operation. Divisions are an ideal structure to deal with large adversaries in vast extensions of land without adjacent superior units or support for their own troops. However, the presence of these capacities makes Divisions difficult and cumbersome to be deployed within a short timeline or in case of emergency; in addition, divisions require a multifaceted C3 to manoeuvre its constituent units.

On the other hand, Brigades are much simpler and lighter structures, easier to be deployed and combat ready to respond to emergencies. However, Brigades are limited in their operational capacity due to their lack of some elements of organic combat support such as heavy artillery or army aviation. Hence, Brigade firepower and manoeuvring capabilities are limited to specific tasks and scenarios, and do not have the possibility of meeting the same responsibilities assigned to a Division.

Nevertheless, the total disbandment of Divisions suggests a lack of proper judgment and planning to comply with Russia's threat and danger assessment. Firstly, although ABs correspond to a modern military concept of light manoeuvrability, and their, albeit limited, capabilities allow their deployment in delimited areas such as the North Caucasus or the borders with Norway and Finland, they cannot be used to counteract effectively larger adversaries such as NATO or China that would most probably operate with Divisions or augmented Task Forces. As argued by Thornton (2011, p. 29) "Brigades, for all their flexibility and speed of response in a complex and compact theatre of operations, could simply be lost in the vast tracts of the land in the Far East".

Secondly, and as a consequence of the first point, in the case of hostile ambitions from a great power, ABs would have the role of slowing down invaders, opting for the usage of nuclear weapons as a deterrent resource. However, this would increase the scale of the confrontation progressively into a larger-scale war, with terrible consequences. In 2010, to mend the gaps in the AB system, it was announced that Brigades would be formed and equipped in three types: light, medium and heavy (McDermott, 2011b, p. 24) with the last of these three resembling the firepower of a Division.

Thirdly, ABs still lack a fundamental element of manoeuvre of a modern military force -Army Aviation- which limits even more its capacity to inflict rapid and decisive strikes or evacuating wounded personnel from the theatre of operations. In fact, the subordination of Army Aviation to the Air Force is one of the biggest mistakes in the Russian military and the continuation of this policy signifies the absence of analysis of the experience of other military forces. Air Forces are usually dominated by a fighter jet/fixed wing (airplane) culture and they despise rotary-wing (helicopter) assets (Thornton, 2011, p. 24). In fact, Army Aviation is an important part of any modern Army as it guarantees tactical manoeuvrability, fire support, medical evacuations, and air assault missions, which are all impossible to perform with airplanes; besides, as the assets are flown by Army pilots, the understanding and commitment to the mission of the Army troops is assured.

Lastly, the inexistence of an Army Unit between Brigades and their MD/USC signifies the undermining of future leadership and command skills of Army Officers. While Colonels command ABs, Colonel Generals command MD/USC, meaning the absence of units with command of troops for Major Generals and Lieutenant Generals. "Only after having the experience as a Division Commander can a General obtain real operational and strategic skill" (Sergunin, 2012, p. 254).

Given the above points, it is possible to conclude that in the organisational area, the 2008 MR has partially prepared Russia's military to manage contemporary global threats and challenges. Russia's MoD has not managed to recognise the disadvantages of establishing a universal solution for the Army units, whereas the best approach may be the organisation and commission of its forces depending on the scenarios and tasks assigned. Figure 9 highlights the elements of RMC that have not been achieved due to the lack of adequate organisation of its forces. It depicts that the lack of an adequate organisation has undermined the process of equipping and training these units, and thus, their mobility and interoperability. Moreover, enhanced capabilities have not been achieved due to their close focus on achieving the basic requirements. Therefore, the capacity to deal with diverse circumstances in different scenarios has been weakened. Further military exercises are required to identify the best structure for the Russian military, however, these practices are expensive and time-consuming. Perhaps the use of theoretical models and war games may offer better options to the high-ranking officials.

Training

Training is a fundamental part of every military as its quality will signify the difference between life and death in the combat areas, and the victory or loss of the battle. Besides, training excellence influences combat readiness, discipline, morale, and respect for the leaders. In the case of Russia, the 2008 MR aimed to improve the training system in four dimensions: firstly, a new intensive initial training for conscripts of three months, instead of the previous six, given that, since 2008, conscripts serve for twelve months only; secondly, the improvement of training for professional soldiers who will be responsible for technical tasks such as being tank drivers or operating howitzers; thirdly, the formation of an educational system for the NCO corps that will be responsible for the tactical leadership and control of conscripts and professional soldiers; fourthly, the improvement of Officer education and training from a Soviet-era system of eight years during their careers to a "western-style system of one stint in a military academy followed by short courses to gain skills needed for specific positions" (Gorenburg, 2013c, pp. 3-4).

Although all of these ideas have been implemented, the expected outcome has not met MoD expectations. Firstly, the poor quality of the conscripts and the combat missions assigned, especially in the North Caucasus, have demonstrated that the three-month training period is insufficient (Dzutsev, 2013, p. 3) and hence, the units are not combat ready. In addition, the permanent incorporation and discharge of conscripts every twelve months signifies the usage of an important part of the senior staff in administrative/training tasks and increases the lack of readiness. "The introduction of sudden alert drills has demonstrated the lack of preparedness in some units, but is likely to lead to an increase in readiness in the long term" (Gorenburg, 2013a).

Secondly, the professional soldiers have proved to be of equally low quality as the conscripts and many have not met the requirements for technical tasks (Herspring and McDermott, 2010, p. 289), hence, the MoD initiated a campaign of alternative conscription for university students, to fill these gaps (RIA Novosti, 2014). Besides, Minister Shoigu had to reverse the elimination of the WO corps, undertaken in the first stages of Serdyukov's tenure, to rehire technical experience dismissed precipitately (Gorenburg, 2013c, p. 4). However, the military is an area of employment that does not attract many candidates due to its low salaries and the poor well-being of its lower members.

Thirdly, the projections for the NCO corps training system have not succeeded in attracting candidates. The programme started in 2009 aiming to enrol 2,000 new candidates each year, however, the first class had only 254 trainees (Herspring and McDermott, 2010, pp. 290-291) and the number of candidates has maintained within the same range across the years. Consequently, the troops are undermanned in NCO terms and professional soldiers are taking tactical command, a task they are not trained for.

Fourthly, officer education was impacted by the closure and consolidation of 65 military institutes for higher education into 10 (later reversed to 16) in 2009 (Nichol, 2011, p. 16), which reduced costs but negatively affected the training system of the military as many institutions had to assume training programmes without having the required capability or staff. Hence, the desired Western-style educational system lacks adequate infrastructure and teaching staff to provide military leaders with modern military thinking (Pravda, 2012b).

In light of the above, it is possible to conclude that in the training area, the 2008 MR has failed to properly prepare Russia's military to manage modern global threats and challenges. Moreover, Soviet-era military thinking remains an important part of the educational curricula; and leaders' education on the FSO concept is still years behind. Figure 10 highlights the elements of RMC that have not been achieved due to the lack of proper training for its forces. It depicts that the lack of adequate training has undermined the process of organising the units and their interoperability. Moreover, enhanced capabilities have not been achieved due to their close focus on achieving the basic requirements. Therefore, the capacity to deal with diverse circumstances, in different scenarios, has been weakened.

Materiel

When Russia's President, Vladimir Putin, stated on 30 October 2001 that "Russia's Military-Industrial Complex (MIC) is archaic and does not correspond to contemporary military-political tasks" (Gidadhubli, 2002, p. 2215), the Russian MIC finally accepted its continuous failure, in terms of modernising a Soviet-dated organisation.

Despite several attempts throughout the final years of the Soviet Union and the initial decade of the emerging Russian Federation, the MIC has continued to display a significant technological lag compared to its global competitors. The lack of modern technologies, research projects, and designated funding has significantly affected the possibility of providing effective solutions to the military. This observation is backed by Roffey's findings (2013, p. 187) on Russian defence research and development (R&D); when concluding that its multiple problems will have an "adverse effect on quality and the ability to supply with high qualified specialists and develop new, high-tech, advanced, innovate weapon systems, or strengthening its prospects of cooperating internationally within a 10-year time frame".

The shortcomings of the Russian MIC have posed a barrier to the modernisation plans of the Russian military when seen in the light of the FSO concept. This means that in addition to the obligation of a legal and rational use of the force, there is a necessity for every state, on acquiring surgical military strike capabilities - where the legitimate target is acquired with minimal collateral damage - to do so with high-technological weapons. Hence, the Russian military has continuously strived for the modernisation of its inventories with new and revolutionary models and not simple improvements of Soviet-era platforms. As Russia is quite far ahead in terms of nuclear defence systems (Kristensen and Norris, 2013; Barnes, 2014; Keck, 2014), but well behind on basic military equipment, new technologies, organisational approaches, policy-making, and research and development are all necessary to optimise the MIC and thus, the Russian military.

After the war with Georgia in 2008, Russia found serious shortcomings in its military equipment. Hence, in 2010, a State Armaments Programme (GPV) was approved to equip the military forces for the period 2011-2020. As per GPV-2020, the military are to have 70 per cent modern weapons and equipment by the year 2020; a total of 20.7 trillion roubles has been allocated for this (Vendil Pallin, 2012, p. 45), but there is still a big question about the MIC's ability to support Russia's needs.

An intense debate has been growing on the GPV-2020, due to the technical content of the armament programme. Whereas the MoD aims to procure new armaments, meaning recently fabricated with the latest technology, the MIC will only have the capability to deliver newly manufactured systems of upgraded designs from the Soviet era, which are technologically outdated. Carlsson, Johan and Westerlund (2013, p. 24) also identify this deficiency when analysing the statements on military materiel of the Russian Major General Yevgeni Ilyin, deputy head of the Ministry of Defence's Main Directorate for International Military Cooperation, in March 2013. Although modern can be associated with new or lately produced equipment, General Ilyin made clear that modern equipment refers to certain generations of systems and thus, for example, T-80 main battle tanks - introduced into service in 1976 - and their modifications were modern, regardless of the time they were fabricated. For example, in 2011, after a publicised reception of a batch of the new T-90C main battle tanks - categorised by the MIC as one of the best tanks in the world - the Russian MoD realised the extent of its technological lag and overpricing, compared with Chinese or NATO models (Herspring and McDermott, 2010, p. 297). The T-90 was just the 17th upgrade of the T-72 main battle tank that entered service in 1971, and the price of one unit (118 million roubles) matches that of three German-built Leopard tanks (McDermott, 2011a, p. 2). Therefore, since late 2011, Russia cancelled all its orders for the T-90 (Pravda, 2012a), and looked forward to ordering the new T-99 main battle tank, expected to enter service in 2020.

Despite the foreseen impact of GPV-2020 on the readiness of Russia's Military Forces, scholars and experts have highlighted the following weaknesses: firstly, although there has been an increase in military expenditure over the years, funds allocated were calculated at current prices, not at the constant prices of the initial year. Therefore, the approved volume is being eroded by inflation (Cooper, 2012, p. 12). Secondly, there is uncertainty about the origin of the funding. During the first three years, 1,500 billion roubles in state-guaranteed credits will support GPV2020, however, interest rates are high and, if not repaid, the debt will be a burden on the budget (Cooper, 2012, p. 13). Thirdly, monopolistic prices used between the Ministry of Defence (MOD) and the MIC include more than the 25 per cent officially accepted profit margin for the companies. The industry has pushed for a continuous prepayment scheme in order to avoid delays, but this has jeopardised the availability of MOD assets for other projects (Oxenstierna and Westerlund, 2013, pp. 7-9). Fourthly, corruption, overpriced items, and single-supplier contracts are still common (Oxenstierna and Westerlund, 2013, p. 12). Fifthly, obsolete technologies are still present within MIC companies. A complete upgrade is necessary to get rid of obsolete machinery, and key high-technology systems usually require micro-components that need to be imported from Europe or the US (Oxenstierna and Westerlund, 2013, p. 16). Sixthly, necessary research and scientific projects have limited funding; companies are still run with outdated organisational structures, demanding double or triple the personnel required for simple tasks, while scientific or research posts are vacant (Oxenstierna and Westerlund, 2013, p. 18). Seventhly, the MIC's skilled personnel are near to retirement age, but young professionals and technicians are reluctant to join defence companies, due to low wages and heavy workloads. Scientists and researchers are still emigrating to take up better-paid posts (Cooper, 2012, p. 23). Eighthly, despite the nationalist rhetoric used by the government, a significant proportion of Russia's military equipment is acquired from non-Russian companies, in Italy, France and Israel (Cooper, 2012, p. 19).

An additional weakness relates to Russia's sales policies on high-technology weapons and technology transfer to other countries. Despite the increase in domestic armament orders, the MIC is highly likely to remain supported by its exports due to the unclear financial sustainability of GPV-2020 for the forthcoming years. Consequently, the MIC will continue its commercial relations with other countries such as China - considered as a possible threat to Russia - offering the same systems as those procured for the Russian military (Rosoboronexport, 2013). As reported by the Stockholm International Peace Research Institute (SIPRI) (2014, p. 3) Russia's weapons' exports increased by 28 per cent during 2004-2008 and 2009-2013 and "more than half of its exports went to just three countries: India, China and Algeria". This sales policy has weaken the operational aspect of the Russian military, as the weapons planned to provide Russia's military with a technological advantage are also possessed by other countries. For instance, during the years 2007-2009, 16 S-300PMU-2 (SA-20B) long-range surface-to-air missile systems - the same used for Russia's missile defence - were sold to China, including technology transfer on detection of incoming missiles (SIPRI, 2013, pp. 67-68; IISS, 2014a, p. 181).

In light of the above, it is possible to conclude that in materiel procurement, the 2008 MR has partially prepared Russia's military to manage contemporary global threats and challenges. Russia's military has make progress on the procurement of military materiel, especially in terms of personal equipment. For instance, the new Ratnik (Warrior) combat equipment, which includes armour vests and modular equipment carriers, has been catalogued as being on the same or an even better level than its Western counterparts such as France's FELIN and the United States' Land Warrior (Nekhai, 2013; RIA Novosti, 2013d). Additionally, the adoption of combat footwear instead of long boots and the replacement of foot traps (portyanki) with socks, have improved troop morale and well-being (Henley, 2013; The Telegraph, 2013). Nonetheless, further advances in procurement by the Russian MoD are necessary, especially on the exigency of research, development, and the designing of new weapon systems. The MIC has the opportunity to propose new systems such as UAVs, high-precision missiles, and drones, to support undermanned units of the Russian military and maximise Russia's strategic advantage. Figure 11 highlights the elements of RMC that have not been achieved by the lack of proper materiel for its forces. It depicts that the absence of adequate equipment has undermined the organisation, mobility, and interoperability of the military. Consequently, combat readiness, morale, and strategic initiative as well as the capacity to deal with diverse circumstances in different scenarios have been undermined.

Leadership

Leadership of the 2008 MR has been a debatable topic since its early beginnings. Not only be-cause of the chaotic relations between the former Minister Serdyukov and the high-level military officers, but because of the methods used to implement the reform at all costs. Every military force has specific characteristics and requirements, based on the country concerned and its goals, its history, experience, means available, and idiosyncrasies; hence, it is highly unlikely that the implementation of foreign methods to improve the military will work efficiently without proper adjustments. This is precisely the task of adequate leadership and proper decision-making regarding the needs of the military.

The 2008 MR is an example of the absence of correct leadership and accurate judgment and planning for the needs of the Russian military. Although the Russian MoD claims that the basis for the reform was an analysis of Western militaries, and specifically, those of the US and NATO, it is hard to understand why important decisions were made without taking into account elementary details. To name a few: the downsizing of the forces did not consider the non-existence in Russia of an active reserve structure such as the National Guard or the Army Reserve in the United States, which maintains permanently trained reserves. In addition, the long period required for enlisting and training professional soldiers, to create an NCO corps, and to educate officers under the FSO concept was not fully considered. As concluded by McDermott (2011a, p. 2), commenting on the 2008 MR leadership, "setting impossible and arbitrarily chosen figures or targets for the reform limits its potential success. Painting by numbers cannot be compared with real work of art, and the reform which is only a loose set of disjointed reform aspirations, suffers from lack of focus on raising quality". Consequently, the Russian military decreased its confidence in the MoD civilians and their relations became more strained.

One of the main tasks of Minister Shoigu is to recover civil-military relations, fostering a cohesive workgroup with sufficient synergy to take correct decisions regarding the modernisation of the military. Initiatives such as the augmentation of civil-military participation in the MoD and the decision to put senior civilian officials in uniform have fostered camaraderie and confidence among the military group.

To conclude, and given the above points, it is possible to take the view that, in the leadership area, the 2008 MR has not prepared Russia's military for managing contemporary global threats and challenges. Figure 12 highlights the elements of RMC that have not been achieved due to the lack of proper leadership. It depicts that the absence of adequate leadership has undermined the organisation, mobility, and interoperability of the military. Consequently, combat readiness, morale, and strategic initiative as well as the capacity to deal with diverse circumstances in different scenarios have been undermined.

Personnel

The personnel administration of the 2008 MR has been full of contradictory and reversed decisions, originating from an absence of both adequate leadership and proper judgment and planning. The main efforts were related to the decreasing of the conscription period, in favour of utilising professional soldiers, hence, improvements in salaries and well-being were planned to attract and hire 499,000 kontraktniky (professional soldiers) by 2017 (Klein and Pester, 2014, p. 3). However, in late 2010, about 110,000 professional soldiers were dismissed on the basis of the increase in conscripts due to the financial constraints (Nichol, 2011, p. 7) and the reduction of kontraktniky required. Surprisingly, in 2011, it was announced again that there was a need for 425,000 professional soldiers (Nichol, 2011, p. 7), a barely achievable goal, due to Russia's demographic limitations for the forthcoming years, which have led the military to an undermanning level of almost 60 per cent (RT, 2012b; RIA Novosti, 2014).

Similarly, the total disbandment of the WO corps in 2009, the failure to establish an NCO corps, and the still necessary but precipitated reduction of the officer corps - from 355,000 in 2007 to 150,000 in 2011, later reversed to 220,000 in 2012 (Barabanov, Makienko and Pukhov, 2012, p. 15) - negatively affected the discipline, leadership and morale of the military, and thus, its combat readiness.

In the same way, decisions on the replacement of military personnel by civilians in key positions such as military doctors and military instructors were made to add experience and technical knowledge to the military. However, specific situations such as the impossibility of forcing civilian doctors to serve in combat units or on submarines (Sieca-Kozlowski, 2012), and the lack of knowledge on military subjects were not taken into account.

Notwithstanding the necessity of downsizing the military in the light of the FSO concept, a closer analysis of the demographic parameters of the five countries with the highest defence expenditure (HDE), depicted in Table 8, and the manpower figures projected by the Russian MoD generate a further debate. When comparing the military personnel as a percentage of total population, Russia substantially surpasses the ratio used by other states (Figure 13), however, if the military force is analysed in the light of the distribution of service members per square kilometre, the figure decreases considerably (Figure 14). These analyses confirm both Russia's high recruitment ratio compared to the population and the drastic personnel reduction beyond the standards used by other states and required to defend the motherland.

Taking into account the above arguments, it is possible to conclude that the 2008 MR has failed to administrate personnel appropriately and has not prepared Russia's military to manage modern threats and challenges. Figure 15 highlights the elements of RMC that have not been achieved due to the lack of sufficient personnel management of its forces. It depicts that the absence of adequate personnel administration has undermined the organisation, training, mobility, and interoperability of the military. Consequently, combat readiness, discipline, morale, and strategic initiative as well as the capacity to deal with diverse circumstances in different scenarios have all been undermined.

Facilities

The downsizing planned by the 2008 MR brought about not only the disbandment of a large number of units but the closure of Army, Navy, and Air Force bases. A significant part of these facilities was just decommissioned and awaits further instructions, while others were sold to help finance the reform. A great deal of criticism came from the military community, as many families living in these units were offered no solution in terms of resettling elsewhere, adding more pressure to the official total of 90,700 servicemen without military housing (Litovkin, 2011). Moreover, military hospitals linked to decommissioned facilities were also closed as part of the restructuring. Besides, some units were transferred, from their original sites, which were then closed, to new garrisons that lacked any primary services, accommodation or training areas. In addition, despite the existence of a large quantity of closed military facilities available to be used, in 2013, the Russian MoD decided to build "100 new defence infrastructure facilities, including airfields, and Army and Navy maintenance and supply bases" (RIA Novosti, 2013f).

Although the Russian MoD stated that they had taken every detail into account when ordering the closure of facilities and the reduction of personnel, it is hard to understand how the federal law, which stipulates, "on the status of servicemen" that "upon reaching a total of 20 years or more of military service, servicemen who have been provided with service housing throughout their service are given their own housing in the location where they elect to reside permanently" (Bartles, 2011, p. 71), was not considered.

In the years before the 2008 MR, the military had no funding to provide housing solutions to the servicemen and as a temporary solution, personnel were authorised to stay in the military, whilst waiting for housing allocation. When the downsizing was ordered, the dismissed servicemen demanded their housing rights, thereby creating a huge hole in the MoD budget. Although a significant number of the military families had no housing solution or military hospital to attend, a large part of the MoD budget for facilities has been expended on the maintenance and upgrading of silo-based ICBMs.

Taking into account the above arguments, it is possible to conclude that the 2008 MR has partially failed to provide appropriate facilities to Russia's military to handle contemporary threats and challenges. Figure 16 highlights the elements of the RMC that have not been achieved due to the lack of sufficient adequate facilities for the forces. It depicts that the absence of sufficient facilities has undermined the organisation, training and interoperability of the military. Consequently, combat readiness, discipline and morale, and the capacity to deal with military dangers and military threats in regional wars, local wars and armed conflicts have been undermined.

Conclusion

Assessing a military reform will always be a multifaceted task, and in the Russian case, there are additional obstacles due to the confidentiality of the topic, diverse opinions of experts and scholars, and the existence of contradictory data. Nevertheless, this paper accomplished the main goal of analysing the 2008 MR while examining conclusions from other studies, adding a new perspective to the debate, and providing a standpoint for further discussions on the topic.

By using an independent method proper to the military sciences, the DOTMLPF assessment, this paper suggested an ideal configuration of Russia's military to meet the national security and defence policy, and military doctrine, in four areas: its basic requirements, its enhanced characteristics, its capacity to deal with a specific challenge and the possible scenarios. Moreover, this paper provided an assessment of how the lack of DOTMLPF variables has impacted negatively on Russia's Military Capabilities (RMC).

While the main negative impact on the basic requirements of RMC (organised, equipped, manned, trained, mobile, interoperable) is made by the lack of an adequate doctrine (Figure 17), the main negative impact on the enhanced characteristics of RMC (combat ready, disciplined, well led, high morale, strategic initiative) is made by the lack of proper leadership (Figure 18). Alternatively, both the focus of its capabilities and the scenarios of operation are affected negatively by the lack of any of the variables (Figure 19 and 20).

This paper demonstrated how two understudied factors are crucial in explaining the failure of the 2008 MR to modernise effectively: firstly, the lack of an adequate leadership; and secondly, the absence of proper judgment and planning for the needs of the Russian military.

These two factors were responsible for undermining Russia's Military Capabilities (RMC) as incorrect decisions prevented an inadequate input of doctrine, organisation, training, materiel, leadership, personnel, and facilities. Consequently, these mistaken strategies and contradictory resolutions left Russia unprepared to handle satisfactorily contemporary and future threats and challenges.

What is more, the points highlighted above aggravated the situation of the Russian military, already affected by budget limitations, systemic corruption, opposition, technological lag, and demographic constraints, and led to a profound revision of the 2008 MR, including the ousting of the Minister of Defence and his advisors, and the reversal of several of their decisions.

In light of the above, it is possible to conclude that the 2008 MR has not been an adequate response to global threats and challenges of the twenty-first century as it has failed to modernise effectively Russia's military and the Military-Industrial Complex (MIC). The 2008 MR requires a structural revision to bring its goals closer to Russia's economic, social, and demographical reality.

As adequate leadership and decision-making are essential to continue with the reform, Minister Shoigu has the challenge of personifying a leader humble enough to take into account his subordinates' views and those of his subject-matter experts, despite their rank, but wise enough to make the best decisions for the troops, despite political or economic interests.

Special attention should be drawn to the budget limitations, corruption, opposition, technological lag, and demographical constraints that affect Russia's military projections nowadays. If the 2008 MR was designed on Western militaries (Nichol, 2011, pp. 5-6) why not use US and NATO experiences to plan new strategies and solve such dilemmas?

Notwithstanding all the problems and inconsistencies of the 2008 MR, Russia's intention to modernise its military should be praised, especially when - as suggested by Thornton (Thornton, 2011, p. 1) - "change is not a common commodity in Russia". Downsizing a gigantic Soviet-era force, accompanied by generations of conservative Officers and defenders, and countless financial, demographical, and historical difficulties, is not a simple task. After ignoring the problem for several years, Russia's top leaders decided to modernise its military effectively so that it may be capable of supporting Russia's political ambitions and threat assessment.

This paper did not aspire to cover in detail every aspect of the 2008 MR because of its vastness and complexity; moreover, due to the novelty of the method used and the constant changes in the reform, the findings of this study can, and need to be, taken further. Additional research on the 2008 needs to be undertaken before drawing definitive conclusions on such a debatable topic.

Pie de página:

1 Count Sergei Witte, Chairman of the Council of Ministers of the Russian Empire.
2 Literature provides two concepts on Russia's Defence Budget. On the one hand, authors such as the Stockholm International Peace Research Institute (SIPRI) and the United States Department of State (DoS) identify such allocations as Military Expenditure (ME). On the other hand, authors such as the International Institute of Strategic Studies (IISS) and the Swedish Defence Research Agency (FOI) refer to such assets as National Defence Expenditure (NDE). Although both approaches are quite similar and comprehensive, ME figures are higher, apparently due to the inclusion of government agencies engaged in military-related projects. This paper includes information of both approaches but will use NDE as its main reference due to the availability of its projections until 2016.
3 State Armament Program: Gosudarstvennaya Programma Vooruzheniya.

References

1. Adamsky, D., (2008). "Through the Looking Glass: The Soviet Military-Technical Revolution and the American Revolution in Military Affairs". Journal of Strategic Studies, 31(2), pp. 257-294.         [ Links ]

2. Andreyeva, S., (2012). What is behind the dismissal of Russian Defense Minister? [online] Available at: <http://voiceofrussia.com/2012_11_06/What-is-behind-the-dismissal-ofRussian-Defense-Minister/> [Accessed 8 Apr. 2014]         [ Links ].

3. Arbatov, A., (1998). Military Reform in Russia: Dilemmas, Obstacles, and Prospects. International Security, 22(4), pp. 83-134.         [ Links ]

4. Barabanov, M., Makienko, M., & Pukhov, R., (2012). Military Reform: Toward the New Look of the Russian Army. [online] Moscow: Valdai Discussion Club. Available at: <http://valdaiclub.com/publication/46540.html> [Accessed 30 Nov. 2013]         [ Links ].

5. Barnes, D., (2014). Russia Fields More Nuclear Weapons, But Expert Says Not to Panic. [online] Nuclear Threat Initiative. Available at: <http://www.nti.org/gsn/article/russia-fields-more-nuclear-warheads/> [Accessed 2 May 2014]         [ Links ].

6. Bartles, C., (2011). Defense Reforms of Russian Defense Minister Anatolii Serdyukov. The Journal of Slavic Military Studies, 24(1), pp. 55-80.         [ Links ]

7. Blank, S., (2011). Russia's Geo-economic Future; The Security Implications of Russia's Political and Economic Structure. The Journal of Slavic Military Studies, 24(3), pp. 351-395.         [ Links ]

8. Blank, S., (2012). Can Russia Reform? Economic, Political, and Military Perspectives. Available at: <http://www.strategicstudiesinstitute.army.mil/pubs/display.cfm?pubID=1111> [Accessed 1 Sep. 2013]         [ Links ].

9. Bouldin, M., (2004). "The Ivanov Doctrine and Military Reform: Reasserting Stability in Russia. The Journal of Slavic Military Studies, 17(4), pp. 619-641.         [ Links ]

10. Bruneau, T., and Matei, F., 2013. The Routledge handbook of civil-military relations. New York: Routledge.         [ Links ]

11. Bryce-Rogers, A., (2013). "Russian Military Reform in the Aftermath of the 2008 Russia-Georgia War". Demokratizatsiya: The Journal of Post-Soviet Democratization, 21(3), pp. 339-368.         [ Links ]

12. Chaykovskaya, E., (2012). Russian army still plagued by violence and corruption despite reforms. [online] The Moscow News. Available at: <http://themoscownews.com/russia/20120209/189440291.html> [Accessed 17 Apr. 2014]         [ Links ].

13. Cheloukhine, S. & Haberfeld, M. (2011). Russian Organized Corruption Networks and their International Trajectories. New York: Springer.         [ Links ]

14. Cimbala, S. (2013). The Russian Military Into the 21st Century. New York, NY: Routledge.         [ Links ]

15. Clover, C. (2009). Moscow marches on with military reform. Financial Times. [online] 25 Jun. Available at: <http://www.ft.com/cms/s/0/e57b3e2c-61a7-11de-9e03-00144feabdc0.html#axzz2ywxAXYtp> [Accessed 15 Apr. 2014]         [ Links ].

16. Cohen, A. (2012). The Downfall of Russia's Defense Minister. The Foundry: Conservative Policy News from The Heritage Foundation. Available at: <http://blog.heritage.org/2012/11/06/the-downfall-of-russias-defense-minister/> [Accessed 7 Apr. 2014]         [ Links ].

17. Cooper, J. (2012). Can Russia afford to modernize its military? [online] Available at: <http://www.sipri.org/research/armaments/milex/publications/unpubl_milex/Cooper%20 Presentation%20SIPRI%202012.pdf> [Accessed 20 Dec. 2013]         [ Links ].

18. DW, (2014). Overall global military spending falls, with increases seen in Russia, Asia and Africa. [online] DW.DE. Available at: <http://www.dw.de/overall-global-military-spending-fallswith-increases-seen-in-russia-asia-and-africa/a-17564643> [Accessed 14 Apr. 2014]         [ Links ].

19. Dzutsev, V. (2013). Putin Cuts Conscript Soldiers' Combat Training Time to Three Months. Eurasia Daily Monitor, 10(30), pp. 2-4.         [ Links ]

20. Eckel, M. (2008). Opposition to Russian military reforms grows. [online] USA Today. Available at: <http://usatoday30.usatoday.com/news/world/2008-11-18-1994976891_x.htm> [Accessed 30 Jun. 2013]         [ Links ].

21. Fater, Y. (2010). Israel signs $400 million deal to sell spy drones to Russia. [online] Haaretz.com. Available at: <http://www.haaretz.com/print-edition/business/israel-signs400-million-deal-to-sell-spy-drones-to-russia-1.318972> [Accessed 19 Apr. 2014]         [ Links ].

22. Felgenhauer, P. (2008). A Military Reform That Does Not Seem to Satisfy Anyone. The Jamestown Foundation, [online] 5(241). Available at: <http://www.jamestown.org/programs/edm/single/?tx_ttnews%5Btt_news%5D=34282&tx_ttnews%5BbackPid%5D=166&no_cache=1> [Accessed 3 Apr. 2014]         [ Links ].

23. Felgenhauer, P. (2009). A Profound Change in the Russian Military may be happening as the power of the General Staff is undermined. Perspective, [online] XIX(1). Available at: <https://www.bu.edu/iscip/vol19/felgenhauer1.html> [Accessed 1 Sep. 2013]         [ Links ].

24. Felgenhauer, P. (2013a). Investigators harass Serdyukov, while military reform is stalled. [online] Jamestown Foundation. Available at: <http://www.ecoi.net/local_link/235763/358599_de.html> [Accessed 9 Apr. 2014]         [ Links ].

25. Felgenhauer, P. (2013b). Is Shoigu reversing Serdyukov's military reform? Russian Military Reform. Available at: <http://russiamil.wordpress.com/2013/02/11/is-shoigu-reversing-serdyukovs-military-reform/> [Accessed 29 Mar. 2014]         [ Links ].

26. Fernandez-Osorio, A. (2015). Full Spectrum Operations: the rationale behind the 2008 Russian Military Reform? Rev. Cient. Gen. Jose Maria Cordova, 13(15), pp. 19-34.         [ Links ]

27. Flintoff, C. (2012). Russian Scandal Hints At Larger Political Battle. [online] NPR.org. Available at: <http://www.npr.org/2012/12/11/166879584/russian-scandal-hints-at-larger-political-battle> [Accessed 10 Apr. 2014]         [ Links ].

28. Gavrilov, Y. (2008). General Reductions: They Will Reduce the Armed Forces and Change their Look. [online] Rossiyskaya Gazeta. Available at: <http://www.rg.ru/2008/10/15/vooruzh-sily.html> [Accessed 12 Apr. 2014]         [ Links ].

29. Gidadhubli, R. (2002). Russia's Military Industrial Complex: Struggle for Revival. Economic and Political Weekly, 37(23), pp. 2215-2218.         [ Links ]

30. Golts, A. (2008). A military spoiler doctrine. [online] Available at: <http://rbth.ru/articles/2008/12/16/161208_military.html> [Accessed 3 May 2014]         [ Links ].

31. Golts, A. (2013). New defense minister at the mercy of conservative generals. [online] Russia Beyond The Headlines. Available at: <http://rbth.com/opinion/2013/01/17/new_defense_minister_at_the_mercy_of_conservative_generals_21963.html> [Accessed 10 Apr. 2014]         [ Links ].

32. Golts, A. (2014). Putin's Way of Reforming the Army. [online] The Moscow Times. Available at: <http://www.themoscowtimes.com/opinion/article/putins-way-of-reforming-the-army/497256.html> [Accessed 3 Apr. 2014]         [ Links ].

33. Gorenburg, D. (2009). Russia's New Model Army: Radical Reform in the Russian Military. [online] Russian Military Reform. Available at: <http://russiamil.wordpress.com/2009/08/14/21/> [Accessed 7 Apr. 2014]         [ Links ].

34. Gorenburg, D. (2013a). Impressions from Moscow. [online] Ponars Eurasia. Available at: <http://www.ponarseurasia.com/article/impressions-moscow> [Accessed 1 May 2014]         [ Links ].

35. Gorenburg, D. (2013b). Serdyukov's Reforms. [Russian Defence Policy] Available at: <http://russiandefpolicy.wordpress.com/category/serdyukovs-reforms/> [Accessed 7 Apr. 2014]         [ Links ].

36. Gorenburg, D. (2013c). The Russian Military under Sergei Shoigu Will the Reform Continue? Available at: <http://www.ponarseurasia.org/sites/default/files/policy-memos-pdf/Pepm_253_Gorenburg_June%202013_0.pdf> [Accessed 1 Sep. 2013]         [ Links ].

37. Gresh, J. (2011). "The Realities of Russian Military Conscription". The Journal of Slavic Military Studies, 24(2), pp. 185-216.         [ Links ]

38. Hansl, B. (2014). Russia Economic Report. [online] Moscow: The World Bank, p. 60. Available at: <http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2014/04/08/000333037_20140408140841/Rendered/PDF/866400WP0P14660RER0No.310FINAL07414.pdf> [Accessed 1 Apr. 2014]         [ Links ].

39. De Hass, M. (2011). Russia's Military Reforms. Victory after Twenty Years of Failure? Clingendael Papers. [online] Netherlands Institute of International Relations. Available at: <http://www.clingendael.nl/sites/default/files/20111129_clingendaelpaper_mdehaas.pdf> [Accessed 24 Jun. 2013]         [ Links ].

40. Hedenskog, J. & Vendil Pallin, C. (eds.) (2013). Russian Military Capability in a Ten-Year Perspective -2013. Available at: <https://www.academia.edu/5407818/2013_Russian_Military_Capability_in_A_Ten -Year_Perspective> [Accessed 13 Dec. 2013]         [ Links ].

41. Henley, J. (2013). Why Russian soldiers are finally replacing foot wraps with socks. The Guardian. [online] 16 Jan. Available at: <http://www.theguardian.com/world/shortcuts/2013/jan/16/russian-soldiers-replacing-foot-wraps-socks> [Accessed 17 Mar. 2014]         [ Links ].

42. Herspring, D. (2008). Russian Military Reform and Anatoly Serdyukov. Problems of Post-Communism, 55(6), pp. 20-32.         [ Links ]

43. Herspring, D., & McDermott, R. (2010). Serdyukov Promotes Systemic Russian Military Reform. Orbis, 54(2), pp. 284-301.         [ Links ]

44. IISS (2008). "Country comparisons - commitments, force levels and economics". The Military Balance, 108(1), pp. 419-450.         [ Links ]

45. IISS (2011a). "Chapter Five: Russia". The Military Balance, 111(1), pp. 173-194.         [ Links ]

46. IISS (2011b). "Chapter Ten: Country comparisons - commitments, force levels and economics". The Military Balance, 111(1), pp. 451-482.         [ Links ]

47. IISS (2012a). "Chapter Five: Russia". The Military Balance, 112(1), pp. 183-204.         [ Links ]

48. IISS (2012b). "Chapter Two: Comparative defense statistics". The Military Balance, 112(1), pp. 31-38.         [ Links ]

49. IISS (2013a). "Chapter Five: Russia and Eurasia". The Military Balance, 113(1), pp. 199-244.         [ Links ]

50. IISS (2013b). Chapter Two: Comparative defense statistics". The Military Balance, 113(1), pp. 41-48.         [ Links ]

51. IISS (2014a). "Chapter Five: Russia and Eurasia". The Military Balance, [online] 114(1), pp. 161-200. Available at: <http://www.tandfonline.com/doi/abs/10.1080/04597222.2014.871878> [Accessed 20 Mar. 2014]         [ Links ].

52. IISS (2014b). "Chapter Two: Comparative defence statistics". The Military Balance, [online] 114(1), pp. 23-30. Available at: <http://www.tandfonline.com/doi/abs/10.1080/04597222.2014.871875> [Accessed 20 Mar. 2014]         [ Links ].

53. Innes, W. (2013). Military Families Will Be Hit Hard By Spending Cuts in 2014. [online] IVN.us. Available at: <http://ivn.us/2013/08/08/military-families-will-be-hithard-by-spending-cuts-in-2014/> [Accessed 12 Apr. 2014]         [ Links ].

54. Interfax (2014). IMF slashes Russian GDP growth forecast in 2014 to 0.2 percent from 1.3 percent, further cuts possible. [online] Russia Beyond The Headlines. Available at: <http://rbth.com/news/2014/04/30/imf_slashes_russian_ gdp_growth_forecast_in_2014_to_02_from_13_further_cu_36345.html> [Accessed 3 May 2014]         [ Links ].

55. Isachenkov, V. (2012). Anatoly Serdyukov, Russia Defense Minister, Dismissed. [online] Huffington Post. Available at: <http://www.huffingtonpost.com/2012/11/06/anatoly-serdyukov-dismissed_n_2080992.html> [Accessed 8 Apr. 2014]         [ Links ].

56. ITAR-TASS (2014). Disruption of contract on Mistral helicopter carriers envisages serious penalty sanctions. [online] ITAR-TASS. Available at: <http://en.itar-tass.com/world/724110> [Accessed 19 Apr. 2014]         [ Links ].

57. Jensen, D. (2013). Will the Prodigal Son Be Forgiven? [online] Institute of Modern Russia. Available at: <http://imrussia.org/en/politics/624-will-the-prodigal-son-be-forgiven> [Accessed 10 Apr. 2014]         [ Links ].

58. Kalinin, I. (2012). Russian Military Reforms. Peace Magazine, 28(3), Jul., p. 13.         [ Links ]

59. Kamenev, M. (2009). How to Dodge the Draft in Russia. Time. [online] 30 Mar. Available at: <http://content.time.com/time/world/article/0,8599,1888238,00.html> [Accessed 21 Apr. 2014]         [ Links ].

60. Keck, Z. (2014). Russia's Military Begins Massive Nuclear War Drill. [online] The Diplomat. Available at: <http://thediplomat.com/2014/03/russias-military-begins-massive-nuclear-war-drill/> [Accessed 1 May 2014]         [ Links ].

61. Klein, M. & Pester, K. (2014). Russia's Armed Forces on Modernisation Course Progress and Perspectives of Military Reform. SWP Comments. [online] Berlin: German Institute for International and Security Affairs. Available at: <http://www.swp-berlin.org/fileadmin/contents/products/comments/2014C09_kle_pst.pdf> [Accessed 9 Apr. 2014]         [ Links ].

62. Kommersant (2007). Civilian Economist Heads Defense Ministry. Kommersant. [online] 16 Feb. Available at: <http://www.kommersant.com/p743268/Anatoly_Serdyukov> [Accessed 12 Apr. 2014]         [ Links ].

63. Kosals, L., and Izyumov, A. (2011). The Russian Defence Industry Confronts the Market: Findings of a Longitudinal Study. Europe-Asia Studies, 63(5), pp. 733-756.         [ Links ]

64. Krainova, N., & Oliphant, R. (2012). Putin Fires Defense Minister Amid Scandal. [online] The Moscow Times. Available at: <http://www.themoscowtimes.com/news/article/putin-fires-defense-minister-amid-scandal/471013.html> [Accessed 8 Apr. 2014]         [ Links ].

65. Kramer, A. (2012). "Putin Fires Defense Minister Linked to Real Estate Investigation". The New York Times. [online] 6 Nov. Available at: <http://www.nytimes.com/2012/11/07/world/europe/putin-dismisses-russian-defense-minister.html> [Accessed 8 Apr. 2014]         [ Links ].

66. Kramer, A. (2013). Russia Charges Ex-Defense Minister Over a Road Project. The New York Times. [online] 28 Nov. Available at: <http://www.nytimes.com/2013/11/29/world/europe/russian-investigators-charge-former-putin-ally.html> [Accessed 7 Apr. 2014]         [ Links ].

67. Kristensen, H. & Norris, R. (2013). Global nuclear weapons inventories, 1945-2013. Bulletin of the Atomic Scientists, 69(75), pp. 75-81.         [ Links ]

68. Kumar, A. (2013). Russian Military Reforms: An Evaluation. IDSA Issue Brief. [online] Available at: <http://www.idsa.in/system/files/IB_RussianMilitaryReforms-AmitK.pdf> [Accessed 10 Dec. 2013]         [ Links ].

69. Lannon, G. (2011). "Russia's New Look Army Reforms and Russian Foreign Policy". The Journal of Slavic Military Studies, 24(1), pp. 26-54.         [ Links ]

70. Liaropoulos, A. (2008). The Russian Defense reform and its limitations. Caucasian Review of International Affairs, 2(1), pp. 42-49.         [ Links ]

71. Litovkin, V. (2010). "Purge". Defense & Security, (25), p. 6.         [ Links ]

72. Litovkin, V. (2011). Solutions to Homeless Colonels. [online] Nezavisimoye Voyennoye Obozreniye. Available at: <nvo.ng.ru/realty/2011-03-18/1_serdyukov.html> [Accessed 16 Mar. 2014]         [ Links ].

73. Litovkin, V. (2012). On the dismissal of Defense Minister Anatoly Serdyukov. [online] Valdai Discussion Club. Available at: <http://valdaiclub.com/defense/51180.html> [Accessed 10 Apr. 2013]         [ Links ].

74. Makarychev, A., and Sergunin, A. (2013). Russian military reform: institutional, political and security implications. Defense & Security Analysis, 29(4), pp. 356-364.         [ Links ]

75. Mankoff, J. (2012). Complex political calculations after Serdyukov's ouster. [online] Valdai Discussion Club. Available at: <http://valdaiclub.com/defense/51800.html> [Accessed 10 Apr. 2014]         [ Links ].

76. McDermott, R. (2011a). The Bear, the Abacus and Impossible Defense Computations. Eurasia Daily Monitor, [online] 8(56). Available at: <http://www.jamestown.org/programs/edm/single/?tx_ttnews%5Btt_news%5D=37679&tx_ttnews%5BbackPid%5D=512&no_cache=1> [Accessed 16 Mar. 2014]         [ Links ].

77. McDermott, R. (2011b). The Brain of the Russian Army: Futuristic Visions Tethered by the Past. [online] Fort Leavenworth, Kansas: The Foreign Military Studies Office. Available at: <http://fmso.leavenworth.army.mil/documents/futuristic-visions.pdf> [Accessed 1 Dec. 2013]         [ Links ].

78. McDermott, R. (2012). The Hedgehog in the Fog: Serdyukov Awaits Putin's Decision. [online] Refworld. Available at: <http://www.refworld.org/docid/4fbdfe5d2.html> [Accessed 10 Apr. 2014]         [ Links ].

79. McDermott, R. (2013). Shoigu Kicks Serdyukov Reform into Long Grass. [online] Refworld. Available at: <http://www.refworld.org/docid/519c7bc04.html> [Accessed 17 Mar. 2014]         [ Links ].

80. Miller, J. (2013). "Growing Corruption In the Russian Military". The Interpreter. Available at: <http://www.interpretermag.com/growing-corruption-in-the-russian-military/> [Accessed 17 Apr. 2014]         [ Links ].

81. National Research University Higher School of Economics (2013). How transparent is Russian Defense sector. [online] Transparency International Russia. Available at: <http://transparency.org.ru/en/news/how-transparent-is-russian-defence-sector> [Accessed 9 Apr. 2014]         [ Links ].

82. Nekhai, O. (2013). Russian troops to receive innovative battle suit 'Ratnik'. [online] The Voice of Russia. Available at: <http://voiceofrussia.com/2013_07_12/Russian-troopsto-receive-new-battle-suit-Ratnik-9990/> [Accessed 29 Mar. 2014]         [ Links ].

83. Nemtsova, A. (2012). Russia's Defense Ministry Crackdown. [online] The Daily Beast. Available at: <http://www.thedailybeast.com/articles/2012/10/25/russia-s-defense-ministry-crackdown.html> [Accessed 10 Apr. 2014]         [ Links ].

84. Nichol, J. (2011). Russian Military Reform and Defense Policy. CRS Report for Congress. [online] Congressional Research Service. Available at: <http://www.fas.org/sgp/crs/row/R42006.pdf> [Accessed 16 Nov. 2013]         [ Links ].

85. Norton-Taylor, R. (2014). "Russia spends more of its wealth on arms than US in 2013". The Guardian. [online] 14 Apr. Available at: <http://www.theguardian.com/world/2014/apr/14/russia-spends-more-wealth-armsus-2013> [Accessed 14 Apr. 2014]         [ Links ].

86. Orr, M. (2000). Better or Just not so Bad? An Evaluation of Russian Combat Effectiveness in the Second Chechen War. Available at: <https://www.google.ru/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0CCwQFjAB&url=http%3A%2F%2Fda.mod.uk%2FCSRC%2Fdocuments%2FRussian%2FD67&ei=3_k6U6uMJvHV4QS1lYDYAw&usg=AFQjCNE3JAwAte0_6fFrJoiDoRp8l83xoA> [Accessed 16 Nov. 2013]         [ Links ].

87. Orttung, R. & Latta, A. (2013). Russia's Battle with Crime, Corruption and Terrorism. New York, NY: Routledge.         [ Links ]

88. Oxenstierna, S. & Westerlund, F. (2013). Arms Procurement and the Russian Defense Industry: Challenges Up to 2020. The Journal of Slavic Military Studies, 26(1), pp. 1-24.         [ Links ]

89. Pravda, (2012a). Russia refuses from T-90 tanks and sells them to Africa. [online] English pravda.ru. Available at: <http://english.pravda.ru/business/companies/14-02-2012/120512-russia_t_90_tank-0/> [Accessed 20 Apr. 2014]         [ Links ].

90. Pravda, (2012b). Russia to close air defense academy to lose billions of profit. [online] English pravda.ru. Available at: <http://english.pravda.ru/russia/politics/30-03-2012/120947-air_defense_academy_tver-0/> [Accessed 6 May 2014]         [ Links ].

91. Pukhov, R. (2008). Anatoly Serdyukov's Iron Hand. Defense & Security, (18), p. 6.         [ Links ]

92. RAPSI (2013). Russian army generals testify against ousted defense minister. [online] RAPSI. Available at: <http://rapsinews.com/anticorruption_news/20130314/266713026.html> [Accessed 10 Apr. 2014]         [ Links ].

93. Renz, B. (2010). Russian Military Reform. The RUSI Journal, 155(1), pp. 58-62.         [ Links ]

94. RIA Novosti (2011). Russia signs $1.7 bln deal for 2 French warships. [online] RIA Novosti. Available at: <http://en.ria.ru/russia/20110617/164684636.html> [Accessed 19 Apr. 2014]         [ Links ].

95. RIA Novosti (2012a). Russia Faces Shortage of Military Draftees. [online] RIA Novosti. Available at: <http://en.ria.ru/military_news/20120118/170819500.html> [Accessed 28 Apr. 2014]         [ Links ].

96. RIA Novosti, 2012b. Russia 'May Buy' $50 Mln Worth of Israeli UAVs. [online] RIA Novosti. Available at: <http://en.ria.ru/military_news/20120516/173495603.html> [Accessed 19 Apr. 2014]         [ Links ].

97. RIA Novosti (2012c). Russian Military Faces Major Reshuffle. [online] RIA Novosti. Available at: <http://en.ria.ru/military_news/20121109/177337098.html> [Accessed 25 Apr. 2014]         [ Links ].

98. RIA Novosti (2012d). Russian Military to Buy 60 Italian Tactical Vehicles. [online] RIA Novosti. Available at: <http://en.ria.ru/military_news/20120124/170926228.html> [Accessed 20 Apr. 2014]         [ Links ].

99. RIA Novosti (2013a). Corruption Up 450% in a Year in Russian Military - Prosecutors. [online] RIA Novosti. Available at: <http://en.ria.ru/crime/20130711/182183954/Corruptionup-450-in-a-Year-in-Russian-Forces--Prosecutors.html> [Accessed 17 Apr. 2014]         [ Links ].

100. RIA Novosti (2013b). Over 240,000 Russian Men Dodged Draft Last Year. [online] RIA Novosti. Available at: <http://en.ria.ru/military_news/20130313/179984970/Over240000-Russian-Men-Dodged-Draft-Last-Year.html> [Accessed 21 Apr. 2014]         [ Links ].

101. RIA Novosti (2013c). Russian Ex Chief of Staff Gets Defense Minister Aide Job. [online] RIA Novosti. Available at: <http://en.ria.ru/military_news/20130304/179804307/Russian-Ex-Chief-of-Staff-Gets-Defense-Minister-AideJob.html> [Accessed 9 Apr. 2014]         [ Links ].

102. RIA Novosti, (2013d). Russian Military to Adopt 'Future Soldier' Gear in 2014. [online] RIA Novosti. Available at: <http://en.ria.ru/military_news/20131020/184248906.html> [Accessed 29 Mar. 2014]         [ Links ].

103. RIA Novosti (2013e). Russia Takes 127th Place in Corruption Survey. [online] RIA Novosti. Available at: <http://en.ria.ru/russia/20131203/185230215/Russia-Takes-127thPlace-in-Corruption-Survey.html> [Accessed 9 Apr. 2014]         [ Links ].

104. RIA Novosti (2013f). Russia to Build 100 New Military Bases and Airfields. [online] RIA Novosti. Available at: <http://en.ria.ru/military_news/20130627/181914590/Russia-to-Build-100-New-Military-Bases-and-Airfields.html> [Accessed 6 May 2014]         [ Links ].

105. RIA Novosti (2013g). Serdyukov's Refusal to Testify Legal -Chief Military Prosecutor. [online] RIA Novosti. Available at: <http://en.ria.ru/russia/20130221/179599593/SerdyukovsRefusal-to-Testify-Legal---Chief-Military-Prosecutor-.html> [Accessed 7 Apr. 2014]         [ Links ].

106. RIA Novosti (2014). Russian Army to Expand Conscription Alternative for Students. [online] RIA Novosti. Available at: <http://en.ria.ru/military_news/20140212/187422622/Russian-Army-to-Expand-Conscription-Alternative-forStudents-.html> [Accessed 21 Apr. 2014]         [ Links ].

107. Roffey, R. (2013). Russian Science and Technology is Still Having Problems-Implications for Defense Research. The Journal of Slavic Military Studies, 26(2), pp. 162-188.         [ Links ]

108. Rosenberg, S. (2012). Putin dismisses defence minister. [online] BBC News. Available at: <http://www.bbc.co.uk/news/world-europe-20218216> [Accessed 8 Apr. 2014]         [ Links ].

109. Rosoboronexport (2013). Rosoboronexport - Export Products Catalogue. [online] JSC Rosoboronexport. Available at: <http://www.roe.ru/mmc/index.html> [Accessed 10 Jan. 2014]         [ Links ].

110. Rotberg, R. (2009). Corruption, Global Security, and World Order. Cambridge, Mass.: Brookings Institution Press.         [ Links ]

111. RT (2012a). Duma moves to ban draft dodgers from state service. [online] Available at: <http://rt.com/politics/ban-draft-dodgers-service-214/> [Accessed 21 Apr. 2014]         [ Links ].

112. RT (2012b). Russian military in urgent need of conscripts. [online] Available at: <http://rt.com/politics/new-russian-outraged-forces-318/> [Accessed 21 Apr. 2014]         [ Links ].

113. RT (2014). Russia surpasses UK as third biggest defense spender - report. [online] RT. Available at: <http://rt.com/business/russia-increases-military-spendings-702/> [Accessed 14 Apr. 2014]         [ Links ].

114. Rusmilitary (2013). Russian Military Higher Educational Institutions - Reverse Reform. Rusmilitary. Available at: <http://rusmilitary.wordpress.com/2013/10/17/russian-military-higher-educational-institutions-reverse-reform/> [Accessed 9 Apr. 2014]         [ Links ].

115. Russian Analytical Digest (2012). Military Reform in Russia. [online] Bremen: The Russian Analytical Digest. Available at: <http://www.isn.ethz.ch/Digital-Library/Publications/Detail/?ots591=0c54e3b3-1e9c-be1e-2c24a6a8c7060233&lng=en&id=146477> [Accessed 10 Nov. 2013]         [ Links ].

116. Russian Federation Ministry of Defense (2010). The Military Doctrine of the Russian Federation. Available at: <http://carnegieendowment.org/files/2010russia_military_doctrine.pdf> [Accessed 14 Mar. 2013]         [ Links ].

117. Russian Federation Ministry of Defense (2013). Mission and Objectives of the Russian Armed Forces. [online] Russian Federation Ministry of Defence. Available at: <http://eng.mil.ru/en/mission/tasks.htm> [Accessed 20 Feb. 2014]         [ Links ].

118. Saradzhyan, S. (2008). Tax collector to head Russia's defense. [online] Centre for Analysis of Strategies and Technologies. Available at: <http://www.cast.ru/eng/?id=306> [Accessed 1 Apr. 2014]         [ Links ].

119. Sergunin, A. (2012). On the Russian Military Reform: A Rejoinder. The Journal of Slavic Military Studies, 25(2), pp. 251-256.         [ Links ]

120. Sieca-Kozlowski, E. (2012). 'An effective system of outsourcing in the Armed Forces has not yet been formed' - E-mail interview with Aleksandr Perendzhiev, Russian Association of Military Politologists, March 2012. The Journal of Power Institutions in Post-Soviet Societies. Pipss.org, [online] (14). Available at: <http://pipss.revues.org/3935> [Accessed 23 Feb. 2014]         [ Links ].

121. SIPRI (2012). SIPRI Military Expenditure Database. [online] Solna, Sweden: Stockholm International Peace Research Institute. Available at: <http://milexdata.sipri.org/files/?file=SIPRI+milex+data+1988-2012+v2.xlsx> [Accessed 12 Dec. 2013]         [ Links ].

122. SIPRI (2013). Transfers of major conventional weapons: suppliers, 2008-2012. [online] Solna, Sweden: Stockholm International Peace Research Institute. Available at: <http://www.sipri.org/databases/armstransfers> [Accessed 13 Feb. 2014]         [ Links ].

123. SIPRI (2014). Trends in International Arms Transfers, 2013. [online] Solna, Sweden: Stockholm International Peace Research Institute. Available at: <http://www.sipri.org/re-search/armaments/transfers/measuring/at-images/The%20five%20largest%20exporters%20of%20major%20conventional%20weapons%20-2004-2008%20and%202009-13-%20and%20their%20recipient%20states%20-2009-13.png> [Accessed 20 Mar. 2014]         [ Links ].

124. Soloviov, V. (2008). "Serdyukov's Purge". Defense & Security, (100), p. 19.         [ Links ]

125. Sukhov, O. (2013). Investigators Open Negligence Case Against Serdyukov | News. [online] The Moscow Times. Available at: <http://www.themoscowtimes.com/news/article/investigators-open-negligence-case-against-serdyukov/490502.html> [Accessed 7 Apr. 2014]         [ Links ].

126. The Economist (2012). "Reform and be sacked". London, UK: The Economist. [online] 10 Nov. Available at: <http://www.economist.com/news/europe/21566006-firing-russias-defence-minister-may-be-setback-military-reform-reform-and-be-sacked> [Accessed 7 Apr. 2014]         [ Links ].

127. The Telegraph (2013). Russian army vows to outlaw 'foot cloths' used instead of socks. [online] Telegraph.co.uk. Available at: <http://www.telegraph.co.uk/news/worldnews/europe/russia/9800768/Russian-army-vows-to-outlaw-foot-clothsused-instead-of-socks.html> [Accessed 17 Mar. 2014]         [ Links ].

128. The World Bank (2012). World Development Indicators. [online] Moscow: The Wold Bank. Available at: <http://databank.worldbank.org/data/views/reports/tableview.aspx> [Accessed 29 Mar. 2014]         [ Links ].

129. The World Bank (2013). Worldwide Governance Indicators. Country Data Report. [online] The Wold Bank. Available at: <http://info.worldbank.org/governance/wgi/index.aspx#countryReports> [Accessed 10 Jan. 2014]         [ Links ].

130. The World Bank (2014). Infographic: Confidence Crisis Exposes Economic Weakness in Russia. [online] Moscow: The Wold Bank. Available at: <http://www.worldbank.org/en/news/feature/2014/03/26/infographic-rer31-confidence-crisis> [Accessed 14 Apr. 2014]         [ Links ].

131. Thornton, R. (2011). Military modernization and the Russian ground forces. [online] Carlisle, PA: Strategic Studies Institute, U.S. Army War College. Available at: <http://www.strategicstudiesinstitute.army.mil/pubs/display.cfm?pubID=1071> [Accessed 30 Dec. 2012]         [ Links ].

132. Transparency International (2013). Corruption Perception Index. [online] Available at: <http://www.transparency.org/country#idx99> [Accessed 9 Apr. 2014]         [ Links ].

133. United Press (2011). IAI delivers 12 UAVs to Russia in key deal. [online] UPI. Available at: <http://www.upi.com/Business_News/Security-Industry/2011/01/17/IAI-delivers12-UAVs-to-Russia-in-key-deal/UPI-57821295280313/> [Accessed 20 Apr. 2014]         [ Links ].

134. U.S. Department of State (2013). World Military Expeditures and Arms Transfers 2013. [online] Washington, D.C.: U.S. Department of State. Available at: <http://www.state.gov/t/avc/rls/rpt/wmeat/2013/index.htm> [Accessed 20 Jan. 2014]         [ Links ].

135. Vendil Pallin, C. (2008). Russian Military Reform: A Failed Exercise in Defence Decision Making. New York, NY: Taylor & Francis US.         [ Links ]

136. Vendil Pallin, C. (ed.) (2012). Russian Military Capability in a Ten-Year Perspective - 2011. FOI-R. Stockholm: Swedish Defence Research Agency.         [ Links ]

137. Wegren, S. & Herspring, D. (eds.) (2009). After Putin's Russia: Past Imperfect, Future Uncertain. Fourth ed. Plymouth: Rowman & Littlefield Publishers.         [ Links ]

138. Weir, F. (2013). Circling the wagons? Putin urges 'drastic upgrade' to Russia's military. Christian Science Monitor. [online] 28 Feb. Available at: <http://www.csmonitor.com/World/Europe/2013/0228/Circling-the-wagons-Putinurges-drastic-upgrade-to-Russia-s-military> [Accessed 9 Apr. 2014]         [ Links ].

139. Weitz, R. (2010). Global Security Watch Russia: A Reference Handbook. Santa Barbara, Calif.: ABC-CLIO.         [ Links ]

140. Weitz, R. (2012). Serdyukov's Fall Puts Russia's Military Reform Under Threat. [online] World Politics Review. Available at: <http://www.worldpoliticsreview.com/articles/12485/serdyukovs-fall-puts-russias-military-reform-under-threat> [Accessed 16 Nov. 2013]         [ Links ].

141. Wengle, S. (2012). Engineers versus managers: experts, market-making and state-building in Putin's Russia. Economy and Society, 41(3), pp. 435-467.         [ Links ]