[Music] welcome to the NASA TLX training session today you will be introduced to and learn more about one of the most commonly used tools in the workplace for workload evaluation this training program will guide you through the crucial components of the nasa TLX and how to use it at any point during the video you can pause rewind and watch section again let's get started [Music] wax on wax off wages to one runway to one right at Echo wind two four zero at three cleared for takeoff hi I'm Staff Sergeant should keep the titular from link talks way north carolina's I'm an air traffic control fundamentals instructor I'm airman first class Richardson from Saint Petersburg Florida and I'm a trainee and air traffic control school you're gonna read be responsible for not only millions and millions of dollars of equipment but a thousands and thousands of lives on your first day of class I teach you that your class is like your crew which would be at your first base you would be on a crew you work as a team it's one unit and everyone needs to help each other get through it come in this room and you're evaluated by an instructor they critique you the entire way so you have to be able to listen to you know what they're saying to you and fixing whatever you just did wrong as well as hear the pilots and remember what they said you just really need to be able to multitask and it's a lot of information at once teaching them the different phraseology is like learning a new language it's the language that we use to communicate with the pilots they're their traffic controllers it's a lot like English but it's it's more compact a little bit more concise go home every night you studied for two hours to practice your phraseology you say it out loud because just say it over in your head isn't going to make you be able to spit it out right when it comes down to it when there's pressure when you're just behind you you just keep saying it over and over and over again until you are going away this two-three Canyon approach Cinna maintain 5,000 turn left heading zero three zero vodka the ILS final course or two on right wait - four zero one zero altimeter - nine or eight by the end of the course you're spinning phraseology Island seven seconds like they told you - you didn't think you'd ever be able to do it but you know you do the best part is is seeing that light bulb come on and even better when you get the emails back saying I got my first rating it's a good feeling today I found out that I'm the top grad for my class and I'll be going to Germany I'm elated I guess this is one of the reasons that I joined the Air Force was to travel it's gonna be a challenge but you know look what I've accomplished so far I'm ready for it does this look easy to you as you heard it involves multitasking understanding technical language working in a time constraint environment and other skills that require focus effort and hours of training how do we know when a task is too much for a worker to complete - overwhelming - stressful or just too complicated is their way of quantifying the subjective feelings of difficult versus easy for a task the National Aeronautics and Space Administration task load index is one way to accomplish this this subjective multi-dimensional assessment tool was published in 1988 by the human performance group at NASA's Ames Research Center over three years and after dozens of laboratory simulations it is a post task questionnaire used to measure workload assessments associated with a given task it is used by designers manufacturers and manages to assess subjective workload in any task in order to improve toss safety and worker performance by identifying workload costs as well as specific factors that influence the received workload of the task since its development it has been applied to many situations most commonly in workplace environments that involve human and machine interfaces such as aviation office workstations or labs in the world of human factors research it is recognized as the gold standard in subjective mental workload assessment and its influence is reflected in its multiple translations and worldwide use let's move on to the context in which the nasa TLX is used we will start with understanding the concept of mental workload mental workload describes the level of attentional resources required to complete a task it can be influenced by task demands external support or past experience modern technology imposes greater cognitive demands upon operators than physical demands therefore understanding how mental workload affects performance is essential the nasa TLX can be used to design jobs that have a lower mental workload and in turn lead to lower stress levels and reduced accident rates in the workplace mental workload is a multi-dimensional concept the main categories of workload measurement are performance based physiological and subjective the NASA plx has been found to be the most valid measurement of subjective workload to have the highest user acceptance to provide sensitive measures of operator load and to have the smallest between subject variability evaluations of operators performance can be difficult and the need to assess subjective mental workload is critical before we begin discussing the specific steps in using the NASA - UX we will break down the components and what they mean while watching the following video think about what could contribute to the stress experienced by the taxi driver keep in mind the ideas of mental demand physical demand temporal demand own performance effort and frustration hi we're gonna hate you oh well then library please okay awesome just let me put on my GPS do you mind if I turn on the music yeah go ahead it's really rainy let me turn on my windshield wipers [Music] won't you let it go yo yeah it was a good time they're really popping it one guy the new the Gangnam style did well I'm really sorry that honestly came out of nowhere I'll take you to your destination now there's a lot of things that I have to pay attention to this rain isn't helping at all we'll come back to this situation after going through the sub-skills in more detail as well as the specific steps in using the nasa TLX these six dimensions can be further divided into three factors that relate to the demands of the task including mental physical and temporal demands the other three scales specifically look at the interaction of the task and the user these include effort frustration and performance when a user is given a NASA 2x package the six scale items are always presented with their definitions this ensures that the user is familiar with the right definitions and there's no variability between users the first subscale is mental demands and is defined as the amount of mental and perceptual activity required for a task was the task easy or demanding simple or complex exacting or forgiving the second subscale physical demands includes the amount of physical activity required for the task such as pushing pulling or controlling the user must think about whether the task was easy or demanding slow or brisk restful or laborious or slack or strenuous temporal demands describes the pressure the individual felt due to the pace at which the task occurred was the pace slow and leisurely or rapid and frantic the fourth item own performance describes how successful the individual felt after completing the task did they accomplish the goals that were previously set and how satisfied were they by their performance effort describes how hard the individual had to work to perform at the level that they did whether it be mentally or physically frustration deals with the feelings of insecurity irritation or discouragement felt during the task it's time for multiple-choice questions which of the following is not a rating scale item is it a frustration the physical demands see cognitive impairment or D own performance if you enter cognitive impairment you are grabbed now that you know how to properly describe the scales how will you administer the nasa TLX to begin the user must read the instructions they must also read the scale definitions to ensure accuracy and complete understanding when the scales are being filled out the next step is for the user to become familiar with the technique of filling out the scales if necessary this can be done by practicing using the scales with sample tasks to ensure the user understands each sub scale if the nasa TLX is being used for research step 3 is when the user performs the task otherwise the task may have already been completed at an earlier time the actual subjective measures come from the next two steps where the user decides the weight of each dimension or factor and later raised each on Anthon asset elx scale the weight of each factor is determined using 15 different comparison cards based on the task just performed the Raider would circle which factor is more important to their workload experience the weights can be obtained anytime after the task is performed next numerical ratings are obtained after the task is completed the user fills out each of the six sub scales and marks an X along the scale whereas it most appropriately fits for the given task all endpoints of the six sub scales are labeled low to high except for the dimension of own performance which is labeled good to poor these ratings measure the magnitude of each of the six dimensions for the task [Music] physical to man man my back was really hurting temporal demands I definitely took a while right there my performance I think I did okay eventually this took a long time I'll put it right there little effort putting quite a bit of effort and Tosh was extremely frustrating that's their perfect to obtain the overall score the 15 comparison cards are tallied and the respective weights determined next the X is marked on the scale are tallied and each is associated with a number from 0 to 100 based on where it falls along the scale this raw rating is combined with the factor weights to give an adjusted reading all six adjusted readings are added together and divided by 15 to give an overall score keep in mind that there are two versions of the NASA TLX the paper version as seen here and the computer version which can be found by following this link both versions will assess mental workload in the same way let's see it in action now we will see an example of how the NASA TLX tool is used in the workplace what are you hey I can dream of all these chances delivery doc please um yeah okay I can do that oh yeah Oh Oh married what's wrong oh I'm just trying to move all these chairs banana and it's really tough they're very happy and I don't know if I can move all of these chairs I'm kind of worried them she's such a million's it done and once I can do it it's tell my father about the chronic low back pain and if I get fired I can't afford that I've got oath off my back but still to pay I'm pretty stressed out oh my god that is a heavy workload hey Mary remember that job I want you to do to move all those chairs off to me yeah I was wondering if you could do a workload analysis [Music] in that video we saw an example of a task that was very physically demanding but not particularly mentally demanding it would be rated appropriately on the NASA TLX what about a more complicated task let's think back to the taxi driver [Music] yeah it was a good time there really what's in it one guy the new via Gangnam style did well I'm really sorry that honestly came out of nowhere I'll take you to your destination now there's a lot of things that I have to pay attention to this brain isn't helping at all what factors would have influenced her mental workload think about this in terms of the six NASA TLX sub-skills we reviewed earlier we'll give you a few seconds to do this [Music] did you consider these stressors the rain the loud radio the pedestrians walking by the GPS the stress of having to reach the destination as quickly as possible all of these factors would contribute to the taxi drivers overall perception of workload let's try it now imagine that you are in the same situation take out a printed copy of the NASA 2x reading scale and fill it out as if you are the taxi driver and your task was to drive the passenger to the destination in that weather think back to what each scale item means and remember to use an X to mark the appropriate spot this is how our taxi driver rated each item how does your nasa 2x compare they may be very different keep in mind that the nasa 2x is a subjective measure of mental workload one person may find that driving with a radio is very distracting and greatly increases overall mental demand whereas another might not perceive it to be a great distraction [Music] as well as being administered to employees or workers the nasa TLX can be used for research for example this study used the nasa TLX as a tool to compare usability of traditional keyboards when compared to touchscreen keyboards such as on an iPad each participant was given the NASA TL X to determine the mental workload of the types of keyboards this is an example of how the NASA 2x can be used by product designers and manufacturers not only to evaluate stress on employees but also to improve their products in the future let's move on to the strength of the NASA TLS we know that NASA TLX is used very often in today's world to measure subjective mental workload but why are so many researchers using this method more often than other methods such as SWAT subjective workload assessment technique and MCH modified Cooper hyper scale a few of the main reasons are because of the superior validity reliability and ease of use of this tool validity is defined as being true under every interpretation and measuring what is intended to be measured in relation to the nasa TLX this means whether or not it is actually mental workload that is being measured for a specific task since NASA TLX is a subjective workload measurement it has to take into consideration different workload measures every answer will be different and many aspects of mwl are measured which increases its validity every person is different and their definitions of underworked and overworked may not be the same therefore a scale is provided the different sub scales allow for the people to access certain areas such as how physically demanding the task was or the effort they put into the task instead of just answering the question what was your mental workload when the subjects see simple questions such as how frustrating was the task they're able to specifically answer them which eventually leads to the more in-depth questions regarding their mental workload NASA TLX is also available in several other languages since translations must be accurate and uniform unintentional bias does not occur further increasing the validity of the NASA TLX being reliable means being able to provide the same results on repeated trials reliability describes how much variance appears in measurements if something is reliable it has minimal errors when it comes to the actual procedure and the results which NASA tilak seems to follow how is NASA TLX reliable first off it can't be modified by the experimenter and in both the computer and online version all questions are the same it is also proven to be the most sensitive when it comes to the very low and very high workload measures when compared to other subjective workload assessments the NASA TLX is preferred by many researchers because of its relatively simple concept the distribution of this workload assessment is done in a quick manner and individuals completing the assessment feel it is easy to understand and fill out because it's available in many mediums it is able to be used in different environments certain researchers are even starting to make the NASA TLX available on iPads and with their own applications this will only increase the popularity of the NASA TLX workload assessment it's time for multiple-choice question why NASA Sheila thought to be better than other workload assessments a and measures high and low workload about its multi-dimensional the user friendly or D all of the above if you enter key I'll be above your crest this concludes the NASA TLX training program so remember if you want to avoid this [Music] use the NASA TLX to identify sources of excessive workload before it becomes an issue for more information visit the nasa TLX official website [Music]