Journal of Sports Science and Medicine
Journal of Sports Science and Medicine
ISSN: 1303 - 2968   
Ios-APP Journal of Sports Science and Medicine
Androit-APP Journal of Sports Science and Medicine
Views
9895
Download
46
 
©Journal of Sports Science and Medicine ( 2015 ) 14 , 16 - 22

Research article
The Relative Age Effect and the Influence on Performance in Youth Alpine Ski Racing
Lisa Müller , Carolin Hildebrandt, Christian Raschner
Author Information
University of Innsbruck, Innsbruck, Austria

Lisa Müller
✉ Department of Sport Science, University of Innsbruck, Fuerstenweg 185, A-6020 Innsbruck, Austria
Email: Lisa.Mueller@uibk.ac.at
Publish Date
Received: 31-07-2014
Accepted: 29-09-2014
Published (online): 01-03-2015
Share this article
 
ABSTRACT

The relative age effect (RAE), which refers to an over representation of athletes born early in a selection year, recently was proven to be present in alpine skiing. However, it was not made apparent whether the RAE exists as early as at the youngest level of youth ski racing at national level, nor whether the relative age influences racing performance. As a consequence, the purpose of the present study was twofold: first, to examine the extent of the RAE and second, to assess the influence the relative age has on the overall performance at the youngest levels of youth ski racing. The study included the investigation of 1,438 participants of the Austrian Kids Cup and 1,004 participants of the Teenager Cup at the provincial level, as well as 250 finalists of the Kids Cup and 150 finalists of the Teenager Cup at the national level. Chi²-tests revealed a highly significant RAE already at the youngest level of youth ski racing (Kids Cup) at both the provincial and national levels. There are not again favorably selected the relatively older athletes from the first into the second level of youth ski racing (Teenager Cup). Among the athletes of the Kids Cup, the relative age quarter distribution differed highly significantly from the distribution of the total sample with an over representation of relatively older athletes by comparison taking the top three positions. The data revealed that relative age had a highly significant influence on performance. This study demonstrated that the RAE poses a problem as early as the youngest level of youth ski racing, thereby indicating that many young talented kids are discriminated against, diminishing any chance they might have of becoming elite athletes despite their talents and efforts. The RAE influences not only the participation rate in alpine skiing, but also the performances. As a result, changes in the talent development system are imperative.

Key words: Birth quarter distribution, ethics, youth ski racing, performance, talent development


           Key Points
  • The relative age influences not only the participation in youth ski racing, but also the performance.
  • The relative age effect is present in all age categories in alpine skiing at national, as well as international level; this indicates that there is a severe loss of talents.
  • From an ethical point of view, the entire talent identification and development process in alpine ski racing is discriminatory against young talented kids; consequently, this process should be reevaluated and changed to reduce the impact of RAE on young alpine ski racers in the future.
  • The system for the competition category classification based on a rotating cut-off-date appears to be an interesting proposal for the reduction of the relative age effect in alpine skiing as well.

INTRODUCTION

The relative age effect (RAE) was first documented in professional Canadian ice hockey (Barnsley et al., 1985); since then researchers have proven its presence in other sports, as well. A RAE exists when the relative age quarter distribution, of a selected sports group shows a biased distribution with more relatively older athletes whose birth months are close to the cut-off date for the competition categories within the selection year, compared to an equal distribution of births months in the general population. (Helsen et al., 2005; Musch and Grondin, 2001)

Up to now, research regarding the RAE has focused on team sports such as soccer (Cobley et al., 2009; Helsen et al., 2005; Romann and Fuchslocher, 2013), basketball (Delorme and Raspaud, 2009; Nakata and Sakamoto, 2011), volleyball (Nakata and Sakamoto, 2011), ice hockey (Delorme et al., 2009; Hurley et al., 2001; Sherar et al., 2007), handball (Schorer et al., 2009), rugby (Till et al., 2014), baseball (Medic et al., 2009), and ekiden (Nakata and Sakamoto, 2011). Individual sports have been investigated less frequently. Nevertheless, a RAE was proven to be present in swimming (Medic et al., 2009), tennis (Edgar and O’Donoghue, 2005), track and field (Medic et al., 2009), and sumo (Nakata and Sakamoto, 2011).

Due to the fact that a RAE is most likely to be present in strength- and power-related sports (Lames et al., 2008), and in culturally popular and important sports (Cobley et al., 2008; Delorme et al., 2009; Helsen et al., 2005), it was hypothesized that a RAE could also exist in alpine ski racing. This hypothesis was proven to be true (Baker et al., 2014; Müller et al., 2012; 2015; Raschner et al., 2012). Alpine ski racing belongs to sports, that on the one hand, require a high level of physical performance (Neumayr et al., 2003; Raschner et al., 2008; Turnbull et al., 2009), and on the other hand, are the most culturally popular sports in several countries (e.g. Austria, Germany, Switzerland) with high selection pressure as a consequence.

A RAE exists in alpine ski racing at international elite and youth levels, as well as at the national youth level. At the international elite level, Baker et al. (2014) found a highly significant RAE in both male and female World Cup ski racers; furthermore, Müller et al. (2012) showed that a highly significant RAE was present in the top elite athletes of the FIS World Cup. At the international youth level, a highly significant RAE was made apparent at the FIS Junior World Ski Championships 2009-2011 with athletes ranging in age from 16 to 20 years. In both studies conducted by Müller et al. (2012), the RAE was found to be present in both genders, in all disciplines and in both European and North American Countries. Additionally, Raschner et al. (2012) found that a highly significant RAE existed among the participants in the alpine ski races at the 1st Winter Youth Olympic Games in Austria in 2012, which included athletes aged 15 to 16 years.

At the national youth level, a highly significant RAE was revealed in the participants of the entrance exams of ski boarding schools in Austria, with athletes aged nine to 10 and 14 to 15 years (Müller et al., 2015). Due to the enormous importance of a high level of physical fitness in alpine ski racing as one key component for success in this sport, Müller et al. (2015) hypothesized that the level of performance of skiing-specific physical motor skills influences the relative age effect, and represents a possible causal mechanism for its existence in alpine ski racing. While no proof supporting this hypothesis resulted from their research, and the factors responsible for the existence of this effect remain unclear, the presence of a RAE prior to the selection at the entrance exams for ski boarding schools was established indicating that the identification of talent and the selection process of the relatively older athletes had already taken place prior to acceptance into the schools (Müller et al., 2015). Therefore, it is of special interest to discover at which point during the talent detection and development process the RAE starts occur in order to be able to change strategies in the talent selection and talent development systems in alpine ski racing. In order to understand the total extent of the RAE in alpine skiing, it is important to assess whether the relative age influences the overall performance in youth ski racing. This said, the aims of the following study were to first examine whether the RAE already exists at the youngest levels of Austrian youth ski racing (the Kids Cup and the Teenager Cup), and second, to assess the influence of the relative age on the overall performance/results in youth ski racing.

METHODS

Participants

In total, 2,878 Austrian youth ski racers (1722♂, 1156♀), who were all members of the Austrian Ski Federation, were examined. All athletes who participated in the Austrian provincial Kids Cup races (n = 1483; 903♂, 580♀), aged 7-11 years, the races at youngest level of youth ski racing in Austria, and those who participated in the Austrian Teenager Cup races (n = 1004; 617♂, 387♀), who were aged 12-15 years, during the 2012/2013 season were considered. Additionally, all the athletes who qualified for the national Kids Cup final races of 2009-2013 (n = 241; 119♂, 122♀), and the national Teenager Championships 2013 (n = 150; 83♂, 67♀), who represented the most successful Austrian ski racers of the given age, were taken into account.

Procedures

The birth dates and the results in the single races were collected; the data were provided by the Austrian Ski Federation (ÖSV). Since the cut-off date for grouping the various competition categories in alpine skiing is the 1st of January, the birth months were split into quarters as follows to calculate relative age: quarter one (Q1) included the months January, February and March; quarter two (Q2), the months April, May and June; quarter three (Q3), the months July, August and September; and quarter four (Q4), the months October, November and December. Data on the relative age quarter distribution of the general population of Austria of the same birth years as the participants in this study (1997-2001) showed an equal distribution of birth months over the four quarters with nearly 25% born in each quarter (www.statistik.at). Therefore, and according to previous research, an equal distribution among the quarters was assumed for the analysis concerning the youth ski racers participating in the Kids Cup races. For the analysis regarding the Teenager Cup participants, the birth quarter distribution of the total sample of the Kids Cup was used as an expected distribution, as proposed by Schorer et al. (2009).

The athletes were divided into four groups according to their personal best results in the Kids Cup or the Teenager Cup: the top three places, places four to 10, places 11-20 and places over 20. This was done in order to evaluate the relative age quarter distribution according to the personal best of each athlete in the races, which would in turn assist in the evaluation of the influence of the relative age on their performance. The athletes who did not finish or who were disqualified were not considered in this calculation.

Statistical analyses

To assess the differences between the observed and the expected equal relative age quarter distribution, chi²-tests (χ²) were used for the total sample and separated for gender, age and competition level. The effect size ω for the chi² tests was calculated. For evaluating the relative age quarter distribution of the participants according to their personal best result throughout χ²-tests, the distribution of the total sample (separated for Kids Cup and Teenager Cup) was used as expected distribution. χ²-tests were also used to assess gender, age and competition level specific differences of the RAE regarding the time difference between the birth month and the month of the cut-off date (January). Odds ratios (ORs) and 95% Confidence Intervals (95% CI) were calculated for relative age quartile distribution according to competition level and gender.

For evaluating the aggregated influence of the relative age on the performance, parameter-free analyses of variance (Kruskal-Wallis-H-Test) were applied (dependent variable: result group in race; independent variable: relative age quarter).

The level of significance was set at p < 0.05 and for highly significant at p < 0.01. All of the calculations were performed using PASW Statistics V.21.0., and the effect size was assessed using G*Power 3.1.9.2. The study was performed according to the Declaration of Helsinki and was approved by the Institutional Ethics Review Board for Human Research of the Department of Sport Science of the University of Innsbruck. Due to the fact that all participants were members of the Austrian Ski Federation, the organization not only provided the data and lent their support for this project, but also took the responsibility for the study.

RESULTS

RAE for Kids Cup and Teenager Cup

The χ²-statistics for the Kids Cup participants showed a highly significant difference from an equal distribution for both the provincial races (χ² = 35.95; p < 0.001; ω = 0.16), as well as the national final races (χ² = 41.67; p < 0.001; ω = 0.42; Figure 1). There was a highly significant competition level specific difference regarding the distance of the birth month to the month of January, with a stronger RAE for the participants of the national final races (χ² = 42.43; p < 0.001). The ORs and the corresponding χ² for each quarter according to competition level are presented in Table 1.

The relative age quarter distributions for the Teenager Cup participants of the provincial races and the National Championships are presented in Figure 2. When compared to the distribution of the Kids Cup participants as expected distribution, the relative age quarter distribution of the participants of the provincial races of the Teenager Cup (χ² = 5.66; p = 0.129; ω = 0.08) did not differ significantly, nor did the distribution of the participants of the National Championships (χ² = 5.6; p = 0.133; ω = 0.21). There was no significant competition level specific difference regarding the distance of the birth month to the month of January within the Teenager Cup. The ORs and the corresponding χ² for each quarter according to competition level are presented in Table 1.

Age- and gender-specific differences in the extent and occurrence of a RAE

No significant age-specific differences were shown in the extent and occurrence of a RAE in the participants of the Kids Cup and Teenager Cup at provincial or at national levels. In addition, there was no significant gender- specific difference in the extent and occurrence of a RAE in the participants of both the Kids Cup and Teenager Cup, neither at the provincial level, nor at the national level. The ORs and the corresponding χ² for each quarter according to gender are presented in Table 1.

Influence of relative age on overall performance

The relative age quarter distribution of the Kids Cup participants divided into the four groups according to their personal best results showed a significantly biased distribution with more athletes of the first relative age quarter only for the top three athletes when compared to the relative age quarter distribution of the total sample (χ² = 9.35; p = 0.025; ω = 0.21; Figure 3), whereas for all the other results groups, no significant differences were shown. The results of the analyses of variance indicated that the relative age had a highly significant influence on the performance (χ² = 22.29; p < 0.001). The ORs and the corresponding χ² for each quarter according to result groups are presented in Table 2.

The relative age quarter distribution for the Teenager Cup participants divided into the four groups according to their personal best results did not show a significant difference between the observed distribution of the four groups and the distribution of the total sample of Teenager Cup participants, but a tendency was shown for the top three athletes (χ² = 7.32; p = 0.062; ω = 0.17; Figure 4). The results of the analyses of variance showed that the relative age had a highly significant influence on the performance (χ² = 12.0; p = 0.007). The ORs and the corresponding χ² for each quarter according to result groups are presented in Table 2.

DISCUSSION

The RAE was investigated previously in alpine ski racing and the findings of the research showed that this effect represents a problem at international elite, as well as at youth and national youth levels. (Baker et al., 2014; Müller et al., 2012; 2015; Raschner et al., 2012) However, whether a RAE already exists at the youngest level of youth ski racing was not investigated. Research into this question is important in order to discover at which point during the talent development system the selection error takes place. Such findings could be fundamental for helping to adapt the process, making it more equitable overall.

The results of this study concluded that the RAE is indeed present in all participants of the youngest level of Austrian youth ski racing, the Kids Cup. A highly significant difference between the expected and the observed relative age quarter distribution was apparent. This indicates that at the youngest level, when the children begin ski racing, a RAE is present. Compared to the relative age quarter distribution of the Kids Cup participants, there was no significant difference found in the distribution of the Teenager Cup participants of the provincial races and the National Championships, although an over representation of athletes born early in the selection year was found. This indicates that at both levels of youth and teenager ski racing (Kids Cup and Teenager Cup) an over representation of relatively older athletes is present. However, compared to the youngest level of youth ski racing, there are not again favorably selected the relatively older athletes into the next level (Teenager Cup), as the percentage of relatively older athletes has not increased from the first to the second level. This shows that the error in the talent selection and development system in alpine skiing already has occurred at or before the youngest level of youth ski racing. This selection error implies that the relatively older athletes are favorably selected, and many talented relatively younger athletes go unnoticed. In this context, the system for the competition category classification based on a rotating cut-off-date by Hurley et al. (2001) in Canadian ice hockey, and modified by Müller et al. (2012) for alpine skiing, is an interesting proposal for reducing and preventing a RAE from occurring. Based on the rotating cut-off date every year, each young ski racer would have the chance to be among every relative age quarter twice throughout the eight years of development in the existing system in Austria, before reaching the qualifiable age of FIS races. This would implement a less biased approach and offer equal opportunities for all young talented ski racers regardless of their birth months.

Athletes who qualified for the national final races of the Kids Cup, showed a highly significant stronger RAE, which indicates that a higher competition level is associated with a stronger RAE at this level of youth ski racing. The descriptive OR revealed that the likelihood of participating at the national final races is 2.79 times higher for an athlete of the first relative age quarter than for one of the third relative age quarter, and even 4.67 times higher for a ski racer of the first relative age quarter compared to one of the last quarter. This is not in line with previous studies such as the study from Schorer et al. (2009) in German handball. They demonstrated that the strength of the RAE decreased as the competition level increased in handball, meaning that early development processes are more germane in the creation of RAEs. However, in handball higher competition levels are accompanied with a higher age of players. This is not the case in the present study, where higher competition levels are associated with athletes selected for ski races at the national level (=higher level) compared to the ski racers of the same age who participate only in provincial races.

In various types of sport, for example in handball and basketball (Delorme et al., 2009), RAEs often do not occur in female contexts; whereas in other sports like soccer (Romann and Fuchslocher, 2013), a RAE is indeed present in female athletes. In the present study, no gender specific differences were observed in the Kids Cup participants. This is in agreement with Baker et al. (2014), Müller et al. (2012; 2015), and Raschner et al. (2012) who also found RAEs for both females and males in top elite alpine ski racers though all four studies found a highly significant stronger RAE among the males. A stronger RAE for the males is often attributed to the higher number of athletes competing for selection leading to higher selection pressure. In Austria, the selection pressure is also high within the female context of elite alpine ski racing.

To the authors’ knowledge, there have been no studies published that investigate the influence of the relative age on alpine ski racing performance with the exception of Raschner et al. (2012), who examined the influence of the relative age on performance, showing that of all the participants of the 1st Winter YOG in Innsbruck in 2012, nearly 30% of the medal winners were of the relative age quarter one (eight relative age quarters were given due to the two eligible age groups and the corresponding four quarters per each year), meaning that nearly 30% of all medal winners were born only in the first three months of the two eligible age years. This is in agreement with the present study where a highly significant influence of the relative age on performance was shown for both the Kids Cup and the Teenager Cup participants. Approximately 35% of the Kids Cup participants who placed in the top three positions were of relative age quarter one and only 13% of quarter four, which demonstrates that the relative age not only influences the participation rate of youth ski racing, but also the results.

As Figure 5 shows, the RAE exists in all age categories of alpine skiing; it is present in the youngest level of national youth ski racing, in ski boarding schools, in the 1st YOG and in the FIS Junior World Ski Championships all the way up to the FIS World Cup. The RAE represents a problem at the national, as well as at the international levels. Due to the fact that talent in a sport does not depend on the birth month (Helsen et al., 2005; Lames et al., 2008), it can be assumed that there is a severe loss of talent as a consequence of the existing RAE in all age categories. From an ethical point of view, the existence of a RAE throughout the entire talent development process indicates that it is biased against young talented kids because the relatively younger athletes have fewer opportunities of reaching the elite level despite their talents and efforts.

CONCLUSION

The results of the present study indicated that the relative age effect is indeed already present at the youngest level of national youth ski racing. The relative age not only influences the participation rate of youth ski racing, but also the performance. Consequently, and according to previous studies, that investigated the existing RAE in all age categories all the way up to the FIS World Cup, the relative age effect represents a serious problem for the talent identification and development process of alpine ski racing. The system for the competition category classification based on a rotating cut-off date could help to reduce this effect.

ACKNOWLEDGEMENTS

The authors would like to express their thanks to Donna Kennedy and Gert Ehn (Austrian Ski Federation) for their kind assistance within the project.

AUTHOR BIOGRAPHY

Journal of Sports Science and Medicine Lisa Müller
Employment: Department of Sport Science of the University of Innsbruck, Austria
Degree: Mag, BSc.
Research interests: Talent development and injury prevention in youth alpine ski racing
E-mail: Lisa.Mueller@uibk.ac.at
 

Journal of Sports Science and Medicine Carolin Hildebrandt
Employment: Department of Sport Science of the University of Innsbruck, Austria
Degree: PhD
Research interests: Injury prevention in alpine ski racing
E-mail: Carolin.Hildebrandt@uibk.ac.at
 

Journal of Sports Science and Medicine Christian Raschner
Employment: Department of Sport Science of the University of Innsbruck, Austria
Degree: Ao. Univ.-Prof. Dr.
Research interests: Talent development in alpine ski racing, training
E-mail: Christian.Raschner@uibk.ac.at
 
 
REFERENCES
Journal of Sports Science and Medicine Baker J., Janning C., Wong H., Cobley S., Schorer J. (2014) Variations in relative age effects in individual sports: Skiing, figure skating and gymnastics. European Journal of Sport Science 14, 183-190.
Journal of Sports Science and Medicine Barnsley R.H., Thompson A.H., Barnsley P.E. (1985) Hockey success and birthdate: The relative age effect. Canadian Association of Health, Physical Education and Recreation 51, 23-28.
Journal of Sports Science and Medicine Cobley S., Baker J., Wattie N., McKenna J. (2009) Annual age-grouping and athlete development. A meta-analytical review of relative age effects in sports. Sports Medicine 39, 235-256.
Journal of Sports Science and Medicine Cobley S., Schorer J., Baker J. (2008) Relative age effects in professional German soccer: A historical analysis. Journal of Sports Sciences 26, 1531-1538.
Journal of Sports Science and Medicine Delorme N., Boiché J., Raspaud M. (2009) The relative age effect in elite sport: The French case. Research Quarterly for Exercise and Sport 80, 336-344.
Journal of Sports Science and Medicine Delorme N., Raspaud M. (2009) The relative age effect in young French basketball players: A study on the whole population. Scandinavian Journal of Medicine and Science in Sports 19, 235-242.
Journal of Sports Science and Medicine Edgar S., O’Donoghue P. (2005) Season of birth distribution of elite tennis players. Journal of Sports Sciences 23, 1013-1020.
Journal of Sports Science and Medicine Helsen W.F., van Winckel J., Williams A.M. (2005) The relative age effect in youth soccer across Europe. Journal of Sports Sciences 23, 629-636.
Journal of Sports Science and Medicine Hurley W., Lior D., Tracze S. (2001) A proposal to reduce the age discrimination in Canadian minor hockey. Canadian Public Policy 27, 65-75.
Journal of Sports Science and Medicine Lames M., Augste C., Dreckmann C., Görsdorf K., Schimanski M. (2008) Der “Relative Age Effect” (RAE): neue Hausaufgaben für den Sport (“Relative Age Effect” (RAE): new homework for the sport). Leistungssport 37, 4-9.
Journal of Sports Science and Medicine Müller L., Müller E., Kornexl E., Raschner C. (2015) The relationship between physical motor skills, gender and relative age effects in young Austrian alpine ski racers. International Journal of Sport Science and Coaching 10, -.
Journal of Sports Science and Medicine Müller L., Raschner C., Kornexl E., Hildebrandt C., Bacik M., Kröll J., Müller E. (2012) Zum Relativen Alterseffekt im internationalen alpinen Skirennlauf (The relative age effect in international alpine skiing). Leistungssport 42, 5-12.
Journal of Sports Science and Medicine Medic N., Young B.W., Starkes J.L., Weir P.L., Groves J.R. (2009) Gender, age and sport differences in relative age effects among US Masters swimming and track and field athletes. Journal of Sports Sciences 27, 1535-1544.
Journal of Sports Science and Medicine Musch J., Grondin S. (2001) Unequal competition as an impediment to personal development: A review of the relative age effect in sport. Developmental Review 21, 147-167.
Journal of Sports Science and Medicine Nakata H., Sakamoto K. (2011) Relative age effect in Japanese male athletes. Perceptual and Motor Skills 113, 570-574.
Journal of Sports Science and Medicine Neumayr G., Hoertnagl H., Pfister R., Koller A., Eibl G., Raas E. (2003) Physical and physiological factors associated with success in professional alpine skiing. International Journal of Sports Medicine 24, 571-575.
Journal of Sports Science and Medicine Raschner C., Müller L., Hildebrandt C. (2012) The role of a relative age effect in the first winter Youth Olympic Games in 2012. British Journal of Sports Medicine 46, 1038-1043.
Journal of Sports Science and Medicine Raschner C., Patterson C., Platzer H.P., Lembert S., Bundesinstitut für Sportwissenschaft (2008) Krafttraining im Nachwuchsleistungssport – Theorie trifft Praxis (Strength training within young athletes–theory meets practice). Kraftleistungen österreichischer Nachwuchskaderathleten im alpinen Skirennsport – Normwerte einer Längsschnittstudie über 10 Jahre (Strength performances of young Austrian alpine ski racers – standard values of a longitudinal study over 10 years). Leipzig. Leipziger Verlagsanstalt GmbH.
Journal of Sports Science and Medicine Romann M., Fuchslocher J. (2013) Influences of player nationality, playing position and height on relative age effects at women’s under-17 FIFA World Cup. Journal of Sport Sciences 31, 32-40.
Journal of Sports Science and Medicine Schorer J., Cobley S., Büsch D., Bräutigam H., Baker J. (2009) Influences of competition level, gender, player nationality, career stage and playing position on relative age effects. Scandinavian Journal of Medicine and Science in Sports 19, 720-730.
Journal of Sports Science and Medicine Sherar L.B., Baxter-Jones A.D.G., Faulkner R.A., Russell K.W. (2007) Do physical maturity and birth date predict talent in male youth ice hockey players?. Journal of Sports Sciences 25, 879-886.
Journal of Sports Science and Medicine Till K., Cobley S., O’Hara J., Cooke C., Chapman C. (2014) Considering maturation status and relative age in the longitudinal evaluation of junior rugby league players. Scandinavian Journal of Medicine and Science in Sports 24, 569-576.
Journal of Sports Science and Medicine Turnbull J.R., Kilding A.E., Keogh J.W.L. (2009) Physiology of alpine skiing. Scandinavian Journal of Medicine and Science in Sports 19, 146-155.
 
 
 
Home Issues About Authors
Contact Current Editorial board Authors instructions
Email alerts In Press Mission For Reviewers
Archive Scope
Supplements Statistics
Most Read Articles
  Most Cited Articles
 
    
 
JSSM | Copyright 2001-2019 | All rights reserved. | LEGAL NOTICES | Publisher

It is forbidden the total or partial reproduction of this web site and the published materials, the treatment of its database, any kind of transition and for any means, either electronic, mechanic or other methods, without the previous written permission of the JSSM.

This work is licensed under a Creative Commons License Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.