Journal of Sports Science and Medicine
Journal of Sports Science and Medicine
ISSN: 1303 - 2968   
Ios-APP Journal of Sports Science and Medicine
Androit-APP Journal of Sports Science and Medicine
Views
5940
Download
153
from September 2014
 
©Journal of Sports Science and Medicine (2006) 05, 208 - 214

Research article
Effects of Aging on Perceived Exertion and Pain During Arm Cranking in Women 70 to 80 YEARS OLD
Alain Groslambert1,2, , Céline C. Grange1,2, Stéphane Perrey3, Jérôme Maire1, Nicolas Tordi1, Jean Denis Rouillon1
Author Information
1 Laboratory of Sport Sciences, UFRSTAPS, Place Saint Jacques, 25030 Besançon Cedex, France
2 Laboratory of Applied Mecanic, 24 chemin de l’épitaphe, 25000 Besançon, France
3 Laboratory of Motor Efficience and Deficience, 700 Av du Pic saint Loup, Montpellier 34090, France

Alain Groslambert
✉ Laboratoire des sciences du sport, Place St Jacques, 25 030 Besançon Cedex, France
Email: alain.groslambert@univ-fcomte.fr
Publish Date
Received: 02-12-2005
Accepted: 08-03-2006
Published (online): 01-06-2006
Share this article
 
ABSTRACT

The aim of this study was to examine the effects of aging on perceived exertion (PE) and perceived arm pain (PaP) at the end of a maximal graded arm test in 70- to 80-year -old women. Twelve healthy young (mean age 22.9 ± 3.3 years), and 12 healthy elderly (mean age 74.6 ± 3.7 years) women performed a maximal graded test (GXT) on an arm crank ergometer until exhaustion. The results revealed no significant difference between both groups concerning PE (p > 0.05; Effect Size = 0.62) and when heart rate (HR) was expressed as a theoretical maximal heart rate (THRmax) (p > 0.05; Effect Size = 0.17). Nevertheless, PaP was significantly lower (p < 0.05; Effect Size = 2.95) in the elderly compared to the young group. In conclusion, these results suggest that, at the end of GXT, PE is not influenced, whereas PaP may be altered by aging of the women tested in the present study. Therefore, it appears difficult to use PaP in these elderly women to regulate exercise intensity during a training program.

Key words: CR 10, elderly, pain, exercise


           Key Points
  • At the end of a maximal graded arm test, perceived exertion is not influenced, whereas perceived arm pain may be altered by aging.
  • It appears difficult to use perceived arm pain in elderly women to regulate exercise intensity during a training program.

INTRODUCTION

The decline in physical and mental ability often associated with increased age in persons has both social and economic implications that affect most countries. Indeed, the maintenance of functional capacity and independence of the older person are beneficial both for the individual and society alike. One method to enhance functioning in old age is physical exercise.

Yet, few methods exist that allow older persons to monitor and regulate exercise intensity without using apparatuses. The individual’s subjective feeling of perceived exertion and arm or leg pain through the use of a simple rating scale does offer an alternative exercise measure to heart rate (HR) monitoring. Studies by Borg and Linderholm, 1967, and Bar-Or et al. (1972) reported a good linearity and high correlation (r = 0.77 to 0.90) between rating of perceived exertion (RPE) and HR. However, it has been shown that maximal HR decreases with age (Borg and Linderholm, 1967; Miller et al., 1985). This would imply that the relationship between RPE and HR also may change with age (Aminoff et al., 1996). Bar-Or et al. (1972) reported in 41- to 61-year-old persons that when comparisons are made at a given exercise intensity, RPE is generally lower in young than in older persons. When comparisons are made at the same relative exercise intensity, no significant difference of perceived exertion was observed between young and 50- to 70-year-old healthy persons (Aminoff et al., 1996; Bengtsson et al., 1977; Sydney and Shephard, 1977).

The effects of aging on perceived exertion in more aged persons (i.e. 70 to 80 years of age) are not well documented, however physical activity plays an important role in maintaining fitness at an adequate level for independent living (Rantanen et al., 1997). In the last 4 years, there have been important advances in the study of effort perception in this age group. Dunbar and Kalinski, 2004 reported that 70-year-old women can accurately use RPE to regulate treadmills walking, stairclimbing and cycling ergometer exercise intensity during a 20-week training program. However, at intensities above 40 % of VO2max an acclimation period is needed. Shigematsu et al., 2004 observed in 75-year-old subjects a significant correlation (r = 0.96) between RPE and oxygen uptake (VO2) during a graded maximal cycling exercise test. The authors concluded that RPE is effective in monitoring exercise intensity in older adults. However, all these studies used only lower limbs exercise to investigate perceived exertion. Because upper limbs exercise have a great impact on everyday activities in elderly persons such as gardening or household working, it may be of interest for health practitioners who prescribe exercise for persons of this age group to know if they are able to assess accurately RPE.

Therefore, the aim of the present study is to examine the effects of aging on perceived exertion and muscular pain in 70- to 80-year-old adults. It was hypothesised that compared to young participants, aging could influence perceived exertion and muscular pain at the end of a maximal graded exercise test. In order to avoid possible gender effects observed in previous studies (Miller et al., 1985; Persson et al., 2000), only women were tested in the present study.

METHODS

Subjects

Twelve healthy young (mean age 22.9 ± 3.3 years), and 12 healthy elderly (mean age 74.6 ± 3.7 years) women volunteered to participate in this study. Elderly women were excluded if they were not between 70 and 80 years, they were mentally handicapped or unable to sign the informed-consent form, or were taking medication which might have interfered with exercise testing. Exclusion criteria also included decompensated congestive heart failure, acute myocarditis, myocardial infarction, unstable agina pectoris, uncontrolled cardiac arrhythmias, severe aortic stenosis, severe hypertension and untreated hypertrophic obstructive cardiomyopathy. The Mini-Mental State Examination (MMSE) of Folstein et al., 1975 was used to evaluate the cognitive state of elderly women. The study plan was accepted by the Local Ethical Committee, and all subjects signed an informed-consent form. None of the subjects were smokers or engaged in a regular physical activity program. Relevant characteristics of the subjects are presented in Table 1.

Materials

Perceived exertion (PE) and perceived arm pain (PaP) of subjects were assessed using the CR-10 of Borg, 1998. Since at low power outputs it is hard to determine exactly what constitutes muscular arm pain and whether it can be distinguished from reports of generalized exertion discomfort (Borg et al., 1985), PE and PaP were assessed only at the end of the last stage of a maximal graded arm test (GXT). At the beginning of the test, subjects were provided with a typewritten set of standardized instructions for the use of the CR-10. Subjects were instructed to give CR-10 values immediately at the end of the GXT. According to the recommendations of Borg (1998), specific instructions for scaling arm pain were given. The persons were invited to evaluate the different kinds of pain and their intensities in relation to certain previous experience of pains.

Procedures

All subjects performed a GXT on an arm crank ergometer (Monark Rehab Trainer, Model 881E, Sweden). This apparatus was calibrated before and after each test. After an initial warm-up period of 3 min, exercise began at a power output of 10W for 2 min, followed by 10W increments every 2 min, until exhaustion, using alternatively an adapted increase of cadence and friction resistance (Franklin, 1985). The cadence was between 50 to 70 rpm (Powers et al., 1984). Subjects were asked to perform the exercise until exhaustion. To confirm that exhaustion was reached, two of the three following criteria had to be met: a drop in arm cranking cadence below 50 rpm, a respiratory exchange ratio value (RER) exceeding 1.0, attainment of 80 % of age predicted maximal heart rate. Multi-channel ECGs (Nihon Kodhen, type 2R-701.VK, Japan) were monitored on-line before and throughout the exercise period. A cardiologist was present to supervise ECG signals for each subject in the elderly group. Pulmonary assessment was performed with a validated (King et al., 1999) portable metabolic measurement system cart (Aerosport KB1-C, Aerosport Inc. Ann Arbor, Michigan, USA). Ventilation volumes are calculated using a flat-plate orifice within an open pneumotachometer. The KB1-C was calibrated immediately prior to each test in the low-flow position according to the manufacturer’s specifications using a 3-liter calibrated syringe (Hans-Rudolph Inc., Kansas City, USA). Expiratory gases were sampled and analysed each 20-s period in order to determine the rate of oxygen consumption (VO2), the rate of carbon dioxide production (VCO2), RER, and minute ventilation (VE). HR was continuously monitored during the tests with a Sport Tester (Vantage N Polar Electro, Finland) and the peak HR (i.e. the highest HR values which could be attained at the end of the GXT) was recorded. According to Shephard, 1998, the theoretical maximal heart rate (THRmax) was calculated as the following: THRmax = 220 - age.

In addition, maximal tolerated power (MTP), (i.e. the highest load in Watts which could be maintained with a constant cadence for 1 min) was registered at the end of the GXT. Corresponding peak VO2, and peak ventilation (VE) were also determined. The Ventilatory Threshold (VT) was assessed from respiratory exchange by three observers using the V-slope method (Beaver et al., 1986). The mean of the two closest values was taken into account for calculating the VT. The VE and HR at VT were determined post hoc.

Statistical analysis

As the data from the present study meet the statistical assumptions for using parametric statistics (i.e. homogeneity of variance and normality of the sample distribution), a paired t test was used to compare the physiological and perceptual responses collected at the end of GXT between the 2 groups (SigmaStat, Jandel Corporation, San Rafael, CA; USA). The statistical power was also calculated and had to be comprised between 0.94 and 0.99 for the sample size used in the present study and the alpha level was set at 0.05 (SigmaStat, Jandel Corporation, San Rafael, CA). Effect Size (ES) was also calculated for each test using Cohen’s (1988) definition of small, medium, and large effect size (ES = 0.20, 0.50, and 0.80, respectively). Statistical significance was accepted at the p< 0.05 level.

RESULTS

Except for the age and height, the statistical analysis showed no significant difference of physical characteristics between both groups (Table 1). It is noteworthy that all elderly women obtained a MMSE score above 24 (mean = 28.1 ± 2.1), attesting that their cognitive functions were not altered. For each group, 100 % of subjects attained at least 2 of the 3 criteria used to determine exhaustion state. All subjects in each group attained at least 80 % of theoretical maximal HR and in all groups 11 subjects out of 12 attained a RER value higher than 1.0. A drop in pedalling cadence below 50 rpm was observed in all subjects of both groups.

Concerning the perceptual responses, a significant difference of PaP (p< 0.05; ES = 2.95) was found (Table 2) and revealed that PaP values of the elderly group (mean = 0.7 ± 1.3) were lower than the young group (mean = 5.0 ± 1.6). In addition, at the end of the GXT, 10 women out of 12 scored 0 for PaP in the elderly group, whereas in the young group none scored 0. However, for PE, no significant difference (p> 0.05; ES = 0.62) was found between both groups.

A significant difference of MTP (p < 0.05; ES = 1.51), peak HR (p < 0.05; ES = 7.61), peak VE (p< .05; ES = 5.37), peak VO2 (L·min-1) (p< 0.05; ES = 5.18), peak VO2 (mL·kg-1·min-1) (p< 0.05; ES = 5.59), VE at the Ventilatory Threshold (p< 0.05; ES = 7.25) and HR at the Ventilatory Threshold (p< 0.05; ES = 4.26) was found between both groups (Table 2). However, when peak HR was expressed as a percentage of maximal theoretical HR, no significant difference was found between both groups (p > 0.05; ES = 0.17). There was also no significant difference in RER between both groups (p > 0.05; ES =0.18).

DISCUSSION

In the present study, we hypothesised that compared with young subjects aging could influence perceived exertion and arm pain in 70 to 80 years women at the end of a maximal graded exercise test. The result of the present study showed that perceived exertion is not influenced by aging at the same relative exercise intensity (Aminoff et al., 1996; Bengtsson et al., 1977; Sydney and Shephard, 1977). The PE values found in the young and elderly groups (mean = 5.9 ± 1.4 and 5.0 ± 1.5 points, respectively) are in line with those usually reported in young subjects (4 to 10 points) at the end of a maximal graded test (Borg, 1998; Wilson and Jones, 1980). However, it is worth noting that one elderly subject rated 2 at the end of the GXT. A previous research (Tordi et al., 1998) has reported that exhaustion during a maximal upper limbs exercise is caused to a great extent by peripheral factors (i.e. blood lactate concentration, blood pH, mechanical strain) and then by cardiopulmonary factors (i.e. heart rate, oxygen uptake, respiration rate, minute ventilation). In the elderly group, the subject who rated 2 had a RER value of 0.87. Therefore it is possible that this subject understood the scale correctly but used a low rating value because her pulmonary responses at the end of the GXT were low. Overall these findings suggest that PE could be used to estimate maximal arm-cranking exercise in elderly women.

However, the main finding is that perceived arm pain may be altered by the aging process for older participants in the context of the present study. The low PaP values found in the elderly group (10 women out of 12 scored 0 at the end of the GXT) could be partially explained by the fact that the elderly women may have a blunted pain perception during exercise. Coldwell and Smith, 1966 reported that the physiological basis of muscular pain sensation during exercise is caused by the localised muscular ischemia when muscle contractions are intense. We may hypothesise that the loss of the sensibility of pain and proprio-receptors caused by sarcopenia, decreases the speed and quality of nervous propagation (Lafratta and Canestrari, 1966). Recently, Reeves et al., 2006 have reported that in elderly subjects, molecular, cellular, nutritional and hormonal mechanisms are at the basis of sarcopenia and are responsible for a progressive deterioration in skeletal muscle size and function. For whole muscle, in addition to changes in neural drive, alterations in muscle architecture and in tendon mechanical properties, exemplified by a reduction in tendon stiffness, have been shown to contribute to this phenomenon. Another hypothesis reported by Boutcher, 2000 is that the elderly persons have degradation in cognitive performance, particularly for perception tasks. It is reported that impaired cerebral circulation caused mainly by aging is mostly associated with reduced cognitive performance (Chodzko-Zajko and Moore, 1994; Fabre et al., 2002). However, our results do not support this hypothesis because all elderly subjects had a MMSE score higher than 24, attesting that aging did not alter their cognitive functions. Another hypothesis is that Borg’s CR-10 and its instructions were not adapted for scaling arm pain in elderly persons. In the elderly group, some subjects rated low PE and particularly PaP values. For example, the subject who rated 2 on the PE scale rated also 0 on the PaP. Therefore, it is possible that in elderly persons more familiarisation before using the CR-10 is necessary in order to estimate accurately their perceptions. However, it is surprising to note that one young women pointed 3 on the PaP scale, although this subject meet 2 of the 3 criteria to obtain exhaustion (a drop in arm cranking cadence below 50 rpm, attainment of 83 % of age predicted maximal heart rate) she had a RER value of 0.93 and rated 4 on the PE. Therefore, it is probable that this subject stopped the GXT before exhaustion. For this reason her PaP was low. Further research investigating the relationship between perceived arm pain and exercise intensity in young and elderly subjects and using other pain-rating scales (e.g. Visual Analog Scale of Price et al., 1983) are encouraged to confirm the results of the present study.

Concerning the physiological responses, the significant lower MTP, peak HR, peak VE, and peak VO2 values observed in the elderly group compared to the young group are in line with previous studies carried out on the effects of aging and physiological responses during exercise (Bengtsson et al., 1977; Shvarz and Reibold, 1990; Sydney and Shephard, 1977). However, when peak HR was expressed as a percentage of the theoretical maximal HR, no significant difference was observed between both groups. Therefore, the percentage of the theoretical maximal HR is more convenient than peak HR, VE, and VO2 to judge exercise intensity in elderly women at the end of a maximal graded arm test, and should be used routinely by health practitioners in reconditioning training programs.

Conclusions

The results of our study revealed that, at the end of a maximal graded arm crank test, perceived exertion is not significantly different between young and elderly women. However compared to the young subjects, perceived arm pain is significantly lower in the elderly group. Consequently, these results suggest that perceived arm pain at the end of a maximal graded arm test may be altered by aging of the women tested for the context of the present study. It appears difficult for health practitioners that use perceived muscular arm pain to prescribe or regulate accurately exercise intensity for elderly women (i.e. may be placing them at risk to prescribe exercise at an intensity that is inappropriate for this population). One possible risk is that exercise prescription could be overestimated and result to muscle and tendon damages but also heart failure. The findings of the present study could have significant implications for these practitioners who are encouraged to use the percentage of the theoretical maximal HR and PE, but not PaP, as a useful tool for monitoring and prescription exercise. Moreover, in elderly persons more familiarisation before using the CR-10 is necessary in order to estimate accurately their perceived exertion. Further investigations carried out on a large experimental sample including male and female elderly persons and in other exercise forms (e.g. swimming, walking, cycling) are encouraged to confirm the results of the present study.

AUTHOR BIOGRAPHY

Journal of Sports Science and Medicine Alain Groslambert
Employment: Lab. of Sport Sciences, and Applied Mechanic, Besançon, France.
Degree: PhD
Research interests: Developmental study of perceived exertion, mental training and cycling.
E-mail: alain.groslambert@univ-fcomte.fr
 

Journal of Sports Science and Medicine Céline C. Grange
Employment: Lab. of Sport Sciences, and Applied Mechanic, Besançon, France.
Degree: PhD
Research interests: Perceived exertion and pain in elderly and paraplegic subjects
E-mail: celinegrange@hotmail.com
 

Journal of Sports Science and Medicine Stéphane Perrey
Employment: Laboratory of Motor Efficience and Deficience, Montpellier, France.
Degree: PhD
Research interests: Exercise and performance testing
E-mail: stephane.perrey@univ-montp1.fr
 

Journal of Sports Science and Medicine Jérôme Maire
Employment: Laboratory of Sport Sciences, Besançon, France.
Degree: PhD
Research interests: Rehabilitation training programs in elderly subjects
E-mail: jemaire@hotmail.com
 

Journal of Sports Science and Medicine Nicolas Tordi
Employment: Laboratory of Sport Sciences, Besançon, France.
Degree: PhD
Research interests: Rehabilitation training programs in elderly and paraplegic subjects
E-mail: nicolas.tordi@univ-fcomte.fr
 

Journal of Sports Science and Medicine Jean Denis Rouillon
Employment: Laboratory of Sport Sciences, Besançon, France.
Degree: PhD, MD
Research interests: Exercise and performance testing
E-mail: jean-denis.rouillon@univ-fcomte.fr
 
 
REFERENCES
Journal of Sports Science and Medicine Aminoff T., Smolander J., Korhonen O., Louhevaara V. (1996) Physical work capacity in dynamic exercise with differing muscle masses in healthy young and older men. European Journal of Applied Physiology 73, 180-185.
Journal of Sports Science and Medicine Bar Or O., Skinner E.R., Burskirk E.R., Borg G. (1972) Physiological and perceptual indicators of physical stress in 41 to 61 year old men who vary in conditioning level and body fatness. Medicine and Science in Sports 4, 96-100.
Journal of Sports Science and Medicine Beaver W.L., Wasserman K., Whipp B.J. (1986) A new method for detecting anaerobic threshold by gas exchange. Journal of Applied Physiology 60, 2020-2027.
Journal of Sports Science and Medicine Bengtsson C., Vedin J.A., Grimby G., Tibblin G. (1977) Work capacity in women in relation to age as judged from a maximal work performance test. Scandinavian Journal of Social Medicine 14, 33-39.
Journal of Sports Science and Medicine Borg G. (1998) Perceived exertion and pain scales. Champaign, Illinois. Human Kinetics Publishers.
Journal of Sports Science and Medicine Borg G., Linderholm H. (1967) Perceived exertion and pulse rate during graded exercise in various age group. Acta Medica Scandinavica 472, 194-204.
Journal of Sports Science and Medicine Borg G., Ljunggren G., Ceci R. (1985) The increase of perceived exertion, aches and pain in the arms, heart rate and blood lactate during exercise on a bicycle ergometer. European Journal of Applied Physiology 54, 343-349.
Journal of Sports Science and Medicine Boutcher S.H., Biddle S. J. H., Fox K.R., Boutcher S. H. (2000) In: Physical activity and psychological well-being. Cognitive performance, fitness, and aging. London. Routledge.
Journal of Sports Science and Medicine Chodzko-Zajko W.J., Moore K.A. (1994) Physical fitness and cognitive functioning in aging. Exercise Sport Sciences Reviews 22, 195-22.
Journal of Sports Science and Medicine Cohen J. (1988) . Statistical power of analysis for the behavioral sciences. N.J. Hillsdale. Lawrence Erlbaum Associates.
Journal of Sports Science and Medicine Coldwell L.S., Smith R.P. (1966) Pain and endurance of isometric muscle contractions. Journal of Engineering Psychology 5, 25-32.
Journal of Sports Science and Medicine Dunbar C.C., Kalinski M.I. (2004) Using RPE to regulate exercise intensity during a 20-week training program for postmenopausal women: a pilot study. Perceptual and Motor Skills 99, 688-690.
Journal of Sports Science and Medicine Fabre C., Chamari K., Mucci P., Mass-Biron J., Prfaut C. (2002) Improvement of cognitive function by mental and /or individual aerobic training in healthy elderly subjects. International Journal of Sports Medicine 23, 415-421.
Journal of Sports Science and Medicine Folstein M.F., Folstein S.E., McHugh P.R. (1975) “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research 12, 189-198.
Journal of Sports Science and Medicine Franklin B.A. (1985) Exercise testing, training and arm ergometry. Sports Medicine 2, 100-119.
Journal of Sports Science and Medicine King G.A., McLaughlin J.E., Howley E.T., Bassett D.R.J., Ainsworth B.E. (1999) Validation of aerosport KB1-C portable metabolic system. International Journal of Sports Medicine 20, 304-308.
Journal of Sports Science and Medicine Lafratta C.W., Canestrari R.E. (1966) A comparison of sensory and motor nerve conduction velocities as related to age. Archives of Physiology and Medicine Rehabilitation 5, 286-90.
Journal of Sports Science and Medicine Miller G.D., Bell R.D., Collis M.L., Hoshizaki T.B. (1985) The relationship between perceived exertion and heart rate of post 50 year-old volunteers in two different walking activities. Journal of Human Movement Study 11, 187-195.
Journal of Sports Science and Medicine Persson A.L., Hansson G.A., Kalliomaki A., Moritz U., Sjolund B.H. (2000) Pressure pain thresholds and electromyographically defined muscular fatigue induced by a muscular endurance test in normal women. Clinical Journal of Pain 16, 155-163.
Journal of Sports Science and Medicine Powers S.K., Beadle R.E., Mangum M. (1984) Exercise efficiency during arm ergometry: effects of speed and work rate. Journal Applied of Physiology 56, 495-499.
Journal of Sports Science and Medicine Price D.D., Mc Grath M.C., Rafii A., Buckingham B. (1983) The validation of visual analogue scales as ratio measures for chronic and experimental pain. Pain 17, 45-56.
Journal of Sports Science and Medicine Rantanen T., Era P., Heikkinen E. (1997) Physical activity and the changes in maximal isometric strength in men and women from 75 to 80 years. Journal American of Geriatry Society 45, 1439-1445.
Journal of Sports Science and Medicine Reeves N.D., Narici M.V., Maganaris C.N. (2006) Myotendinous plasticity to ageing and resistance exercise. Experimental Physiology , -.
Journal of Sports Science and Medicine Shephard R.J., Fahey T.D. (1998) Aging and exercise. Encyclopedia of Sports Medicine Science. Internet Society of Sport Sciences , -.
Journal of Sports Science and Medicine Shigematsu R., Ueno L.M., Nakagaichi M., Nho H., Tanaka K. (2004) Rate of perceived exertion as a tool to monitor cycling exercise intensity in older adults. Journal of Aging and Physical Activity 12, 3-9.
Journal of Sports Science and Medicine Shvarz E., Reibold R.C. (1990) Aerobic fitness norms for males and females aged 6 to 75 years: a review. Avion Space and Environmental Medicine 61, 3-11.
Journal of Sports Science and Medicine Sydney S.H., Shephard R.J. (1977) Perception of exertion in the elderly, effects of aging, mode of exercise and physical training. Perceptual and Motor Skills 44, 999-1010.
Journal of Sports Science and Medicine Tordi N., Gimenez M., Predine E., Rouillon J.D (1998) Effects of an interval training programme of the upper limbs on a wheelchair ergometer in able-bodied subjects. International Journal of Sports Medicine 19, 408-414.
Journal of Sports Science and Medicine Wilson R.C., Jones P.W. (1990) Long-term reproducibility of Borg scale estimates of breathlessness during exercise. Clinical Sciences 80, 309-312.
 
 
 
Home Issues About Authors
Contact Current Editorial board Authors instructions
Email alerts In Press Mission For Reviewers
Archive Scope
Supplements Statistics
Most Read Articles
  Most Cited Articles
 
  
 
JSSM | Copyright 2001-2020 | All rights reserved. | LEGAL NOTICES | Publisher

It is forbidden the total or partial reproduction of this web site and the published materials, the treatment of its database, any kind of transition and for any means, either electronic, mechanic or other methods, without the previous written permission of the JSSM.

This work is licensed under a Creative Commons License Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.