Emergency Contact Information

In the event of a system outage, please call us in the following order from the top down:

Mike Russiello  Mobile: +1 571 213-5677
                           Home:   +1 703 938 1615
                           Skype:    mikerussiello

Shoa Appelman   Mobile: 703 966 2080
                            Skype: shoaonskype

Bill Lake:            Office: 703-687 6212
                            Mobile: 202 701 8166
                            Home:  703 266-1427
                            Skype:  williamhlake

Call anytime, day or night if there is an issue.
 




             

Information Regarding Translation and Localization

Translation Overview

At the highest level, the basic steps for localizing HR Avatar tests are:

1. Translate text from inside the test
2. Regenerate images that include text from the test
3. Translate native text for IMO operation
4. Translate text keys for test administration system
5. Translate text keys for test scoring, report generation and distribution system
6. Identify text from tests that is used in animation script and record audio voiceovers.
7. Re-animate the animations used in the test module using the new voiceovers. 
8. Post all text changes. 
9. Check for quality assurance.

The HR Avatar authoring system has an online capability to support all text translation tasks. An HR Avatar administrator creates each translation task and assigns it to an individual translator. The translator then logs on and clicks the translation tab to view all assigned translation tasks. Then the translator clicks on the task they want to work on and begins filling in translated text in the text boxes as appropriate, saving work occasionally.

Because of the modular and template-driven nature of the HR Avatar simulation authoring system, the translation of the first test requires the translation of a set of templates and short test modules. All of these templates and modules are then re-used in all additional test translations so later translations are much easier. The same is true for the HR Avatar test administration, IMO, and report generation/distribution systems. These are needed to support the first test but are all re-used for future tests.

A Collection of Modules

Each HR Avatar Job-Specific test is really a collection of one or more of the following test modules:

15 cognitive simulation modules
2  Personality modules
2 biodata inventory modules
1 essay module
1 typing speed/accuracy module
35+ skills and knowledge modules

Typically, a job specific test uses one cognitive simulation module, one personality module, the essay module, one biodata module, and 0-2 skills/knowledge modules.

Additionally, the test uses several modules that define the beginning, end, and transitions between other modules, or specify templates for interactions and the player look/feel. These modules are used across all tests.

A list of which tests use which module is available on request. 

Note that only the cognitive simulation modules and several of the template modules use animations. All of the other modules are text-based.

In order to complete a test in another language, all modules that are used by that test must be translated.

Planning your first translation

Step 1:  Choose the test or tests that you want to translate. This should be a short list. HR Avatar will then set up the necessary translation tasks to support these tests.
Step 2: Complete all translation jobs.
Step 3: Obtain any images that must be converted to the target language.
Step 4:  Obtain a list of voiceovers that must be recorded for different characters from HR Avatar.
Step 5: Record voiceovers and send to HR Avatar.

Estimating workload:

Templates:
Autogeneration Sim:6000 words
Simlet Template (interaction patterns):  2200 words
Sim Template: 1000 words
Splash In:     400 words, 1 voiceover
Splash Out:  100 words, 1 voiceover
Transition: 25 words
Pre-Personality (intro): 140 words, 1 voiceover
Pre-Biodata (intro): 50 words, 1 voiceover
Pre-Essay (intro): 200 words, 1 voiceover
Pre-Knowledge (intro): 50 words, 1 voiceover

Total templates:  10,200 words, 6 voiceovers

Software Keys:
IMO Operations Keys:  1500 words
Test Administration Keys:  4500 words
Report Generation Keys:   3500 words

Total software keys: 9500 words

Test Modules:
Typical Cognitive Module:  4500 words, 25 voiceovers (30 secs each)
Personality Module:  4500 words
Biodata module:  1000 words
Knowledge/Skills Module: 800 words

Total for first full test modules: 10,800 words, 25 voiceovers

Total for first full test: 30,500 words, 31 voiceovers

After first test complete, maximum for another test is 10,800 words and 25 voiceovers. However, could be no effort also, as long as all the modules are already translated.

Remember, there are only 15 cognitive modules - each test uses one of these. We are creating a few more. Also, there are only two personality and two biodata modules and one essay module.

About voiceovers:

Typically we hire non-union voice actors to come to our office for voice sessions. We have a setup with a good microphone and baffle boards that works with our computer. The 30 voice scripts are probably from about 5 different voice actors. Each person will probably take an hour or so to record.

We can provide additional coaching on creating a voiceover setup. There are also online resources.

How HR Avatar Computes Norms

Revised 2-2-2016

By default, the system attempts to calculate three different norm values for each test score:

Overall - across all valid scores for this test.
Country - across all valid scores for this test where the test-taker was within same country as the current test-taker (based on IP Address).
Account - across all people within a given account who have taken a test.

Here's how each of these three percentiles is calculated:

1. Count the number of valid test events for this test for the current test version id. If there are 25 or more valid events, use this group to calculate the percentile.  Otherwise:

2. Add progressively older test version ids to the pool until the total exceeds 25 valid events. If we find 25 or more, use this group to calculate the percentile.Otherwise:

3. Count the number of valid test events for this test for all test version ids. If there are 25 or more valid events, use this group to calculate the percentile. Otherwise:

4. No percentile is reported.
 
All norms are calculated as a percentile - that is the percent of test events that have a score at or below the current test taker.

Since norms are calculated based on the number of valid test events AT THE TIME OF THE CALCULATION, values can change. For this reason the function "Recalculate Norms" is available in the detail test result section of the test.

Translation Jobs

The HR Avatar Authoring System has the ability to perform online content translation.

The basic process is:

1. An HR Avatar Administrator creates a "Translation Job" on the authoring tool, assigning it to a specific user on the system. 
2. The assigned user logs on to the authoring tool, clicks on the Translations Tab, then chooses a translation job from the list of assigned translation jobs. 
3. The user replaces text in the appropriate boxes with translated text, then clicks save to save all translations.
4. The user indicates the approximate percent complete of the translation job and enters any notes or questions in the notes section.
   When the job is complete, the HR Avatar Administrator places the translation job in complete status and finalizes the changes. 

There are three types of Translation Jobs:

1. IMO
2. Sim
3. Resource Bundle

From the perspective of the translator, these all work the same way. For HR Avatar administrators, the IMO and Sim translation jobs write directly to the target Sim and IMOs.  However, the Resource Bundle translation jobs require IT to import the completed keys when complete.

An IMO Translation Job is designed to translate all the appropriate text components of a specific IMO. A Sim Translation Job translates all the appropriate text components of a specific Sim. Note that this is for the Sim only, and does not include the underlying IMO. Name, competencies, score text, and interview questions are examples of components translated by Sim Translation jobs. Not that Sim Templates, Simlets, and Simlet Templates are never directly assigned to translation jobs. Instead, these should all be translated as IMO translation jobs.

Resource Bundle Translation Jobs are to translate a series of name=value keys that are used by the software in various places, such as the testing system, report generation system, or the direct IMO software.

Translator Guidelines

For all types of translation jobs, the system tries to collect all text snippets that need to be translated, only presenting the components that should be translated.

When you open a translation job, you have these things to do:

1. Modify/replace the text in all the translation text boxes with translated text. 
2. Update the percent complete value. 
3. Enter notes as you have questions, suggestions, or other notes. 
4. Save your work!

 Please keep the following guidelines in mind as you are translating:

• Text that is between curly brackets {camera moves left} in the interaction item or clip segment content is never presented to a test taker and doesn't need to be translated. 
• Square brackets with text in all caps inside, like [FIRSTNAME], should never be translated. 
• Preserve HTML and never change anything inside an HTML tag except the title and alt attributes.
• Cue points ((0.1)) inside Clip Segment content indicate the start of a line of text in the closed captions and should be preserved. After voiceovers are redone and new audio/visual media is completed, these will likely be changed, but they should be kept during the translation phase.

HR Avatar Administrator Guidelines

Note that an underlying Sim or IMO cannot be modified when it is attached to a translation job.

To translate a simple IMO (no Simlet, or other sim object attached):

1. Copy the IMO and rename the copy as the translated version of the IMO. Be sure to change the language of the translated IMO. 
2. Create a translation job in the admin area of the IMO type. Name it, assign the translator, assign the IMO numbers for source and target IMO. Set the language and then make the status "Active."
3. When the job is complete, change the translation job status to complete.

To translate a simlet:

1. Copy the IMO and be sure to copy the simlet also. 
2. Change the language of the copied IMO and copied simlet to the target language. 
3. Create a translation job in the admin area of the IMO type. Name it, assign the translator, assign the IMO numbers for source and target IMO. Set the language and then make the status "Active."
4. When the job is complete, change the translation job status to complete. 
5.  Be sure that the english names of each competency are preserved. This is critical for scoring.

To translate a sim template:

1. Copy the IMO and be sure to copy the sim template also. 
2. Change the language of the copied IMO and copied sim template to the target language. 
3. Create a translation job in the admin area of the IMO type. Name it, assign the translator, assign the IMO numbers for source and target IMO. Set the language and then make the status "Active."
4. When the job is complete, change the translation job status to complete.

To translate a simlet template: Same as sim template.

To translate a Sim:

1. translate all simlets, sim template, and simlet templates using instructions above. 
2. Copy the sim and rename and change the language of the copy to the target language. 
3. Create a translation job in the admin area of the Sim type. Name it, assign the translator, assign the Sim numbers for source and target Sim. Set the language and then make the status "Active." Note that the system always uses the latest version of a sim.
4. When the job is complete, change the translation job status to complete.

HR Avatar Software/Services Pricing

Most partners will sell our off-the-shelf test content. However, we also offer several software products as well as services.

Software User Agreements
$15,000 US/year for the base authoring tool for five users and unlimited IMOs.
$1,500/year for each extra user
$7,500 per year for Sim Builder - a significant productivity enhancer if you are building a lot of simulations


 Support
$1,500/year for premium support (training plus we drop everything to help with problems)

HR Avatar Test Administration



We can host tests, score them, and do everything in the background. We have exceptional capabilities in this area, since the tests are client-side objects. For instance, we can embed a test in a job application as easily as you can put an image on a web page. This can be very exciting to some clients.

All pricing is custom depending on requirements. We will give you a quote and you can mark that up.

HR Avatar Custom Test or Training Creation

All pricing is custom depending on requirements. We will give you a quote and you can mark that up.


Legacy ClicFlic Administrations - used for training objects and demos and things. Not test administration

1. Non-hosted delivery (you put on your own web server):  No Charge

2. Hosted Delivery of content - hosted on our webservers (again, not tests - no scoring):
$100 for 10,000 view events of an IMO. Base product comes with 50,000 view events.

Creating Custom Competencies within our tests

We have the ability to create customized test modules that can be bundled with the standard assessments as custom modules you can embed into the standard assessment. 

It's not trivial, but it's doable, particularly for a large client!

If bundled, the new competencies can be calculated as part of the overall score and totally integrated into the test. However, to do this, we need the items, item scores, and a set of additional information:

1. Competency Description
2. Score Text - this is text that appears in the report and online system.
3. Interview Question - for the report
4. Score Parameters
5. Score Cutoffs

Below is a full example for the competency: Exhibits a Positive Work Attitude.

Description:
For some people, work is a second-place activity. That is, given a decision to take either personal time or go to work, low scorers will choose time off. Low job priority could indicate a 9 to 5 mentality. Sample item: It is OK to take long lunches and breaks if you are underpaid.

Score Text: 

Green Level: 
Expects to receive both financial and personal rewards in exchange for applying his or her best energies to the job. Enjoys working. Trusts the organization to help career.

Yellow-Green Level:
Usually expects to receive both financial and personal rewards in exchange for solid and consistent effort on the job. Enjoys most work activities and is willing to put in extra effort when warranted or requested.

Yellow Level:
Dedicates a medium level of effort to his or her work but is willing to allocate extra effort when asked and rewarded for doing so. Typically neutral regarding job satisfaction and/or enjoyment. Usually trusts management, but can become skeptical when prompted by unexpected circumstances.

Red-Yellow Level:
Sees work as a secondary activity. Most of the time does not enjoy working and often applies average effort at best. Tends to be skeptical regarding management decisions and company policies.

Red Level:
Sees work as a secondary activity to be mostly endured. Dedicates just enough energy to get by. May distrust the organization to assist with his or her career.

Interview Question:

Question:
How do you feel having a regular job and going to work? Is it something you enjoy? Why or why not?

High Anchor:
Views work as a means of income only. Does not enjoy. Does not care about professional reputation.

Medium Anchor:
Likes work but doesn't truly enjoy it. Balances priority and energy with other obligations. 

Low Anchor:
Considers work a key priority in life. Enjoys working and always applies best energy. Takes pride in work reputation.

Scoring Parameters and scoring method:



First, sum the individual item scores for all items in this scale.
Next, use this loookup table to convert the total for each scale to a 0-100 scale:

Total,0-100 Score
11,5,
13,10,
17,15,
20,20,
21,35,
22,45,
23,50,
24,55,
25,60,
26,65,
27,70,
29,75,
30,80,
31,85,
32,90,
34,95,
36,100


Note: A lookup table is not required. We can also use total percent correct, for instance. Please contact us on how you want to score the competency. This is just an example.

Score Cutoffs:

Green: 80 - 100
Yellow-Green: 60-80
Yellow: 40 - 60
Red-Yellow: 20 - 40
Red: 0 - 20

How We Calculate Scores

NOTE: SCORING METHOD CHANGED 7/1/2018. THE FOLLOWING ENTRY IS NO LONGER VALID. PLEASE REFER TO THE NEW METHOD BLOG ENTRY. 

This post is divided into two sections,

• Individual Competency Score Calculations
• Overall Score Calculation

All scores are presented on a 0-100 scale.

Keep in mind that Overall Score is calculated to be essentially a Normal Distribution with a mean of 50 and a standard deviation of 21.  It is not a percentile.

To get the overall score, we must first obtain scores for each individual competency. We then combine these scores in order to arrive at the overall score.

Step 1: Individual Competency Score Calculation

1. For cognitive modules, we calculate the score as the percent of available points the test-taker achieved. 
2. For AIMS, we sum the points from the indiv items (likert scale), apply a lookup table to convert to 0-100.
3. For Biodata, we use the average points per item and then apply a lookup table to convert to 0-100.
4. For skills/knowledge tests, we calculate the percent of available points.

Overall Score Calculation

The overall raw score is the weighted average of the individual competency scores. The overall final score is a standardized, then transformed version of the raw score.

The basic steps are:

1. Determine the score value to use for each competency. In many cases it's the competency score, but not all.
2. Set the weights for each competency using a standard formula.
3. Adjust weights if needed.
4. Apply ONET to the weights.
5. Calculate raw score as weighted average of individual competency scores
6. Standardize the raw score to a z score (mean=0, standard deviation=1)
   
7. Transform the z score to a scale with a mean of 50 and a standard deviation of 21.06

Step 1: Determine the score value to use for each competency.

Some competencies, such as behavioral history and some attitudes, interests, and motivations are set to use the assigned color category rather than the numeric value for that competency. This is because the more desirable regions of the score scale may be in different parts of the scale. For instance, Develops Relationships is best in the middle of the scale rather than at the high end. Or, Past Behaviors may be skewed highly to the high end of the scale.

Step 2. Set standard weights for each competency such that:
  - total  weights for all Cognitive Ability competencies = 1
  - total  weights for all Skills/Knowledge competencies = 0.8
  - total weights for all AIMS competencies  = 0.7
  - Total weights for all Biodata competencies  = 0.4

For individual competencies within a category, the weights are spread evenly (i.e. 1/n each).

Step 2. For some jobs, we add some weight to specific competencies. For instance, we increase the weight on the Expressive AIMS factor for sales people. This weight will override the weight determined in step 1.

Step 3. Next, apply ONET importance ratings to each competency. Since not all competencies have an onet importance value, those that don't are assigned an onet importance of 4.2 (80% on a 1-5 ONET importance scale). This is to prevent these from getting extra weight just because they are not in ONET. Most ONET weights are below this value and the average is about 2.5. The ONET value is converted to a fraction and applied to the weight from steps 1 and 2.  Weight fm step 1 * ONET as fraction = final weight.

Step 4. The overall raw score is the weighted average of the individual competency scores.

Step 5. An overall z score is calculated using a mean/std derived from existing data (if there are over 100 test events in dbms)  for that test or "all tests" if there is not enough data.  The derived mean measured on about 100 events after changing the above is 54.6, std=9.15  These are the values I'm using.

Step 6. The overall z-score is transformed to a score using a mean of 50 and a std of 21.06.  These values are the values used for a Normal Curve Equivalent Scale (NCE) used in various achievement tests.  If the score is outside the range of 0-100, it is set to 0 or 100 as appropriate.

Notes:

1. Key values are the mean and standard deviation of the overall raw score (the weighted average), which is used to create the z score. These values are based on test results for this test. If there are not enough completed results for a given test, overall mean and standard deviation values are used.

2. Sometimes the overall score will look higher than it should be, or lower than it should be. This is because the overall score is not just a raw weighted average, which most people assume. It's a reflection of the number of standard deviations from the mean that the person scored. So, if their overall weighted average is 40, but the average overall weighted average is 80 and the standard deviation is 20, then they are two standard deviations below the mean. So when converted to NCE (mean 50 and std 21.06), they will have an overall score of 50 - 2*21.06 = 7.88. So, the overall score will say 8.
 
Note: For information on the Normal Curve Equivalent Scale (NCE) go to:
http://en.wikipedia.org/wiki/Normal_curve_equivalent