These public domains are needed in order to fully access HR Avatar services:
All Functions
https://vi.hravatar.com
https://cdn.hravatar.com
http://www.hravatar.com
https://www.hravatar.com
https://test.hravatar.com
http://test.hravatar.com
https://s3.amazonaws.com
http://cfmedia.clicflic.com
Testing Only:
https://cdn.hravatar.com
https://test.hravatar.com
http://test.hravatar.com
https://s3.amazonaws.com
Video Interviewing Only:
https://www.hravatar.com
https://cdn.hravatar.com
https://s3.amazonaws.com
Admin Website Only:
http://www.hravatar.com
https://www.hravatar.com
https://s3.amazonaws.com
http://cfmedia.clicflic.com
Custom Test Authoring:
https://sim.hravatar.com
https://imo.hravatar.com
https://s3.amazonaws.com
http://cfmedia.clicflic.com
Certain customers may need to open these domains on their firewall in order to permit access.
Thursday, June 18, 2015
Monday, June 1, 2015
Expired Test Keys and Credits
Credits
Credits are charged to a test taker when they START a test. This means they did something to start the test - typically clicking on a test key link provided by a company and they advance through the name/email or demographic screens to start the test.
Test Key Expiration Dates
All test keys have expiration dates. This is to keep the database clean and avoid having too many test keys that are in limbo, which could degrade the testing experience for legitimate test takers.
Whenever you create a test key - you assign an expiration date. By default this is 3 months after the test key is created. For single-use test keys, it can be changed in the advanced settings when creating the test key or when reviewing the detail for incomplete test keys. For multi-use test keys, the expiration is always 3 months after the test taker starts the test.
When do Test Keys Expire
1. If they have not started the test before the expiration date, the test key will expire on the expiration date and they will not be able to restart the test.
2. If they have started the test before the expiration date, the test key will expire 7 days after the expiration date.
Credits and Expiration
If a test key is expired before it is started by the candidate (expire date), no credits are charged.
If a test key is expired after it was started by a candidate (expire date + 60 days), credits are charged and are not recoverable.
Credits are charged to a test taker when they START a test. This means they did something to start the test - typically clicking on a test key link provided by a company and they advance through the name/email or demographic screens to start the test.
Test Key Expiration Dates
All test keys have expiration dates. This is to keep the database clean and avoid having too many test keys that are in limbo, which could degrade the testing experience for legitimate test takers.
Whenever you create a test key - you assign an expiration date. By default this is 3 months after the test key is created. For single-use test keys, it can be changed in the advanced settings when creating the test key or when reviewing the detail for incomplete test keys. For multi-use test keys, the expiration is always 3 months after the test taker starts the test.
When do Test Keys Expire
1. If they have not started the test before the expiration date, the test key will expire on the expiration date and they will not be able to restart the test.
2. If they have started the test before the expiration date, the test key will expire 7 days after the expiration date.
Credits and Expiration
If a test key is expired before it is started by the candidate (expire date), no credits are charged.
If a test key is expired after it was started by a candidate (expire date + 60 days), credits are charged and are not recoverable.
Monday, April 20, 2015
Emergency Contact Information
In the event of a system outage, please call us in the following order from the top down:
Mike Russiello Mobile: +1 571 213-5677
Home: +1 703 938 1615
Skype: mikerussiello
Shoa Appelman Mobile: 703 966 2080
Skype: shoaonskype
Bill Lake: Office: 703-687 6212
Mobile: 202 701 8166
Home: 703 266-1427
Skype: williamhlake
Call anytime, day or night if there is an issue.
Mike Russiello Mobile: +1 571 213-5677
Home: +1 703 938 1615
Skype: mikerussiello
Shoa Appelman Mobile: 703 966 2080
Skype: shoaonskype
Bill Lake: Office: 703-687 6212
Mobile: 202 701 8166
Home: 703 266-1427
Skype: williamhlake
Call anytime, day or night if there is an issue.
Thursday, April 2, 2015
Information Regarding Translation and Localization
Translation Overview
At the highest level, the basic steps for localizing HR Avatar tests are:
Because of the modular and template-driven nature of the HR Avatar simulation authoring system, the translation of the first test requires the translation of a set of templates and short test modules. All of these templates and modules are then re-used in all additional test translations so later translations are much easier. The same is true for the HR Avatar test administration, IMO, and report generation/distribution systems. These are needed to support the first test but are all re-used for future tests.
A Collection of Modules
Each HR Avatar Job-Specific test is really a collection of one or more of the following test modules:
15 cognitive simulation modules
2 Personality modules
2 biodata inventory modules
1 essay module
1 typing speed/accuracy module
35+ skills and knowledge modules
Typically, a job specific test uses one cognitive simulation module, one personality module, the essay module, one biodata module, and 0-2 skills/knowledge modules.
Additionally, the test uses several modules that define the beginning, end, and transitions between other modules, or specify templates for interactions and the player look/feel. These modules are used across all tests.
A list of which tests use which module is available on request.
Note that only the cognitive simulation modules and several of the template modules use animations. All of the other modules are text-based.
In order to complete a test in another language, all modules that are used by that test must be translated.
Planning your first translation
Step 1: Choose the test or tests that you want to translate. This should be a short list. HR Avatar will then set up the necessary translation tasks to support these tests.
Step 2: Complete all translation jobs.
Step 3: Obtain any images that must be converted to the target language.
Step 4: Obtain a list of voiceovers that must be recorded for different characters from HR Avatar.
Step 5: Record voiceovers and send to HR Avatar.
Estimating workload:
Templates:
Autogeneration Sim:6000 words
Simlet Template (interaction patterns): 2200 words
Sim Template: 1000 words
Splash In: 400 words, 1 voiceover
Splash Out: 100 words, 1 voiceover
Transition: 25 words
Pre-Personality (intro): 140 words, 1 voiceover
Pre-Biodata (intro): 50 words, 1 voiceover
Pre-Essay (intro): 200 words, 1 voiceover
Pre-Knowledge (intro): 50 words, 1 voiceover
Total templates: 10,200 words, 6 voiceovers
Software Keys:
IMO Operations Keys: 1500 words
Test Administration Keys: 4500 words
Report Generation Keys: 3500 words
Total software keys: 9500 words
Test Modules:
Typical Cognitive Module: 4500 words, 25 voiceovers (30 secs each)
Personality Module: 4500 words
Biodata module: 1000 words
Knowledge/Skills Module: 800 words
Total for first full test modules: 10,800 words, 25 voiceovers
Total for first full test: 30,500 words, 31 voiceovers
After first test complete, maximum for another test is 10,800 words and 25 voiceovers. However, could be no effort also, as long as all the modules are already translated.
Remember, there are only 15 cognitive modules - each test uses one of these. We are creating a few more. Also, there are only two personality and two biodata modules and one essay module.
About voiceovers:
Typically we hire non-union voice actors to come to our office for voice sessions. We have a setup with a good microphone and baffle boards that works with our computer. The 30 voice scripts are probably from about 5 different voice actors. Each person will probably take an hour or so to record.
We can provide additional coaching on creating a voiceover setup. There are also online resources.
At the highest level, the basic steps for localizing HR Avatar tests are:
- Translate text from inside the test
- Regenerate images that include text from the test
- Translate native text for IMO operation
- Translate text keys for test administration system
- Translate text keys for test scoring, report generation and distribution system
- Identify text from tests that is used in animation script and record audio voiceovers.
- Re-animate the animations used in the test module using the new voiceovers.
- Post all text changes.
- Check for quality assurance.
Because of the modular and template-driven nature of the HR Avatar simulation authoring system, the translation of the first test requires the translation of a set of templates and short test modules. All of these templates and modules are then re-used in all additional test translations so later translations are much easier. The same is true for the HR Avatar test administration, IMO, and report generation/distribution systems. These are needed to support the first test but are all re-used for future tests.
A Collection of Modules
Each HR Avatar Job-Specific test is really a collection of one or more of the following test modules:
15 cognitive simulation modules
2 Personality modules
2 biodata inventory modules
1 essay module
1 typing speed/accuracy module
35+ skills and knowledge modules
Typically, a job specific test uses one cognitive simulation module, one personality module, the essay module, one biodata module, and 0-2 skills/knowledge modules.
Additionally, the test uses several modules that define the beginning, end, and transitions between other modules, or specify templates for interactions and the player look/feel. These modules are used across all tests.
A list of which tests use which module is available on request.
Note that only the cognitive simulation modules and several of the template modules use animations. All of the other modules are text-based.
In order to complete a test in another language, all modules that are used by that test must be translated.
Planning your first translation
Step 1: Choose the test or tests that you want to translate. This should be a short list. HR Avatar will then set up the necessary translation tasks to support these tests.
Step 2: Complete all translation jobs.
Step 3: Obtain any images that must be converted to the target language.
Step 4: Obtain a list of voiceovers that must be recorded for different characters from HR Avatar.
Step 5: Record voiceovers and send to HR Avatar.
Estimating workload:
Templates:
Autogeneration Sim:6000 words
Simlet Template (interaction patterns): 2200 words
Sim Template: 1000 words
Splash In: 400 words, 1 voiceover
Splash Out: 100 words, 1 voiceover
Transition: 25 words
Pre-Personality (intro): 140 words, 1 voiceover
Pre-Biodata (intro): 50 words, 1 voiceover
Pre-Essay (intro): 200 words, 1 voiceover
Pre-Knowledge (intro): 50 words, 1 voiceover
Total templates: 10,200 words, 6 voiceovers
Software Keys:
IMO Operations Keys: 1500 words
Test Administration Keys: 4500 words
Report Generation Keys: 3500 words
Total software keys: 9500 words
Test Modules:
Typical Cognitive Module: 4500 words, 25 voiceovers (30 secs each)
Personality Module: 4500 words
Biodata module: 1000 words
Knowledge/Skills Module: 800 words
Total for first full test modules: 10,800 words, 25 voiceovers
Total for first full test: 30,500 words, 31 voiceovers
After first test complete, maximum for another test is 10,800 words and 25 voiceovers. However, could be no effort also, as long as all the modules are already translated.
Remember, there are only 15 cognitive modules - each test uses one of these. We are creating a few more. Also, there are only two personality and two biodata modules and one essay module.
About voiceovers:
Typically we hire non-union voice actors to come to our office for voice sessions. We have a setup with a good microphone and baffle boards that works with our computer. The 30 voice scripts are probably from about 5 different voice actors. Each person will probably take an hour or so to record.
We can provide additional coaching on creating a voiceover setup. There are also online resources.
Monday, March 30, 2015
How HR Avatar Computes Norms
Revised 2-2-2016
By default, the system attempts to calculate three different norm values for each test score:
Overall - across all valid scores for this test.
Country - across all valid scores for this test where the test-taker was within same country as the current test-taker (based on IP Address).
Account - across all people within a given account who have taken a test.
Here's how each of these three percentiles is calculated:
1. Count the number of valid test events for this test for the current test version id. If there are 25 or more valid events, use this group to calculate the percentile. Otherwise:
2. Add progressively older test version ids to the pool until the total exceeds 25 valid events. If we find 25 or more, use this group to calculate the percentile.Otherwise:
3. Count the number of valid test events for this test for all test version ids. If there are 25 or more valid events, use this group to calculate the percentile. Otherwise:
4. No percentile is reported.
All norms are calculated as a percentile - that is the percent of test events that have a score at or below the current test taker.
Since norms are calculated based on the number of valid test events AT THE TIME OF THE CALCULATION, values can change. For this reason the function "Recalculate Norms" is available in the detail test result section of the test.
By default, the system attempts to calculate three different norm values for each test score:
Overall - across all valid scores for this test.
Country - across all valid scores for this test where the test-taker was within same country as the current test-taker (based on IP Address).
Account - across all people within a given account who have taken a test.
Here's how each of these three percentiles is calculated:
1. Count the number of valid test events for this test for the current test version id. If there are 25 or more valid events, use this group to calculate the percentile. Otherwise:
2. Add progressively older test version ids to the pool until the total exceeds 25 valid events. If we find 25 or more, use this group to calculate the percentile.Otherwise:
3. Count the number of valid test events for this test for all test version ids. If there are 25 or more valid events, use this group to calculate the percentile. Otherwise:
4. No percentile is reported.
All norms are calculated as a percentile - that is the percent of test events that have a score at or below the current test taker.
Since norms are calculated based on the number of valid test events AT THE TIME OF THE CALCULATION, values can change. For this reason the function "Recalculate Norms" is available in the detail test result section of the test.
Monday, February 16, 2015
Translation Jobs
The HR Avatar Authoring System has the ability to perform online content translation.
The basic process is:
An IMO Translation Job is designed to translate all the appropriate text components of a specific IMO. A Sim Translation Job translates all the appropriate text components of a specific Sim. Note that this is for the Sim only, and does not include the underlying IMO. Name, competencies, score text, and interview questions are examples of components translated by Sim Translation jobs. Not that Sim Templates, Simlets, and Simlet Templates are never directly assigned to translation jobs. Instead, these should all be translated as IMO translation jobs.
Resource Bundle Translation Jobs are to translate a series of name=value keys that are used by the software in various places, such as the testing system, report generation system, or the direct IMO software.
Translator Guidelines
For all types of translation jobs, the system tries to collect all text snippets that need to be translated, only presenting the components that should be translated.
When you open a translation job, you have these things to do:
HR Avatar Administrator Guidelines
Note that an underlying Sim or IMO cannot be modified when it is attached to a translation job.
To translate a simple IMO (no Simlet, or other sim object attached):
To translate a Sim:
The basic process is:
- An HR Avatar Administrator creates a "Translation Job" on the authoring tool, assigning it to a specific user on the system.
- The assigned user logs on to the authoring tool, clicks on the Translations Tab, then chooses a translation job from the list of assigned translation jobs.
- The user replaces text in the appropriate boxes with translated text, then clicks save to save all translations.
- The user indicates the approximate percent complete of the translation job and enters any notes or questions in the notes section.
When the job is complete, the HR Avatar Administrator places the translation job in complete status and finalizes the changes.
- IMO
- Sim
- Resource Bundle
An IMO Translation Job is designed to translate all the appropriate text components of a specific IMO. A Sim Translation Job translates all the appropriate text components of a specific Sim. Note that this is for the Sim only, and does not include the underlying IMO. Name, competencies, score text, and interview questions are examples of components translated by Sim Translation jobs. Not that Sim Templates, Simlets, and Simlet Templates are never directly assigned to translation jobs. Instead, these should all be translated as IMO translation jobs.
Resource Bundle Translation Jobs are to translate a series of name=value keys that are used by the software in various places, such as the testing system, report generation system, or the direct IMO software.
Translator Guidelines
For all types of translation jobs, the system tries to collect all text snippets that need to be translated, only presenting the components that should be translated.
When you open a translation job, you have these things to do:
- Modify/replace the text in all the translation text boxes with translated text.
- Update the percent complete value.
- Enter notes as you have questions, suggestions, or other notes.
- Save your work!
- Text that is between curly brackets {camera moves left} in the interaction item or clip segment content is never presented to a test taker and doesn't need to be translated.
- Square brackets with text in all caps inside, like [FIRSTNAME], should never be translated.
- Preserve HTML and never change anything inside an HTML tag except the title and alt attributes.
- Cue points ((0.1)) inside Clip Segment content indicate the start of a line of text in the closed captions and should be preserved. After voiceovers are redone and new audio/visual media is completed, these will likely be changed, but they should be kept during the translation phase.
HR Avatar Administrator Guidelines
Note that an underlying Sim or IMO cannot be modified when it is attached to a translation job.
To translate a simple IMO (no Simlet, or other sim object attached):
- Copy the IMO and rename the copy as the translated version of the IMO. Be sure to change the language of the translated IMO.
- Create a translation job in the admin area of the IMO type. Name it, assign the translator, assign the IMO numbers for source and target IMO. Set the language and then make the status "Active."
- When the job is complete, change the translation job status to complete.
- Copy the IMO and be sure to copy the simlet also.
- Change the language of the copied IMO and copied simlet to the target language.
- Create a translation job in the admin area of the IMO type. Name it, assign the translator, assign the IMO numbers for source and target IMO. Set the language and then make the status "Active."
- When the job is complete, change the translation job status to complete.
- Be sure that the english names of each competency are preserved. This is critical for scoring.
- Copy the IMO and be sure to copy the sim template also.
- Change the language of the copied IMO and copied sim template to the target language.
- Create a translation job in the admin area of the IMO type. Name it, assign the translator, assign the IMO numbers for source and target IMO. Set the language and then make the status "Active."
- When the job is complete, change the translation job status to complete.
To translate a Sim:
- translate all simlets, sim template, and simlet templates using instructions above.
- Copy the sim and rename and change the language of the copy to the target language.
- Create a translation job in the admin area of the Sim type. Name it, assign the translator, assign the Sim numbers for source and target Sim. Set the language and then make the status "Active." Note that the system always uses the latest version of a sim.
- When the job is complete, change the translation job status to complete.
Friday, February 6, 2015
HR Avatar Software/Services Pricing
Most partners will sell our off-the-shelf test content. However, we also offer several software products as well as services.
Software User Agreements
$15,000 US/year for the base authoring tool for five users and unlimited IMOs.
$1,500/year for each extra user
$7,500 per year for Sim Builder - a significant productivity enhancer if you are building a lot of simulations
Support
$1,500/year for premium support (training plus we drop everything to help with problems)
HR Avatar Test Administration
We can host tests, score them, and do everything in the background. We have exceptional capabilities in this area, since the tests are client-side objects. For instance, we can embed a test in a job application as easily as you can put an image on a web page. This can be very exciting to some clients.
All pricing is custom depending on requirements. We will give you a quote and you can mark that up.
HR Avatar Custom Test or Training Creation
All pricing is custom depending on requirements. We will give you a quote and you can mark that up.
Legacy ClicFlic Administrations - used for training objects and demos and things. Not test administration
1. Non-hosted delivery (you put on your own web server): No Charge
2. Hosted Delivery of content - hosted on our webservers (again, not tests - no scoring):
$100 for 10,000 view events of an IMO. Base product comes with 50,000 view events.
Software User Agreements
$15,000 US/year for the base authoring tool for five users and unlimited IMOs.
$1,500/year for each extra user
$7,500 per year for Sim Builder - a significant productivity enhancer if you are building a lot of simulations
Support
$1,500/year for premium support (training plus we drop everything to help with problems)
HR Avatar Test Administration
We can host tests, score them, and do everything in the background. We have exceptional capabilities in this area, since the tests are client-side objects. For instance, we can embed a test in a job application as easily as you can put an image on a web page. This can be very exciting to some clients.
All pricing is custom depending on requirements. We will give you a quote and you can mark that up.
HR Avatar Custom Test or Training Creation
All pricing is custom depending on requirements. We will give you a quote and you can mark that up.
Legacy ClicFlic Administrations - used for training objects and demos and things. Not test administration
1. Non-hosted delivery (you put on your own web server): No Charge
2. Hosted Delivery of content - hosted on our webservers (again, not tests - no scoring):
$100 for 10,000 view events of an IMO. Base product comes with 50,000 view events.
Thursday, February 5, 2015
Creating Custom Competencies within our tests
We have the ability to create customized test modules that can be bundled with the standard assessments as custom modules you can embed into the standard assessment.
It's not trivial, but it's doable, particularly for a large client!
If bundled, the new competencies can be calculated as part of the overall score and totally integrated into the test. However, to do this, we need the items, item scores, and a set of additional information:
Description:
For some people, work is a second-place activity. That is, given a decision to take either personal time or go to work, low scorers will choose time off. Low job priority could indicate a 9 to 5 mentality. Sample item: It is OK to take long lunches and breaks if you are underpaid.
Score Text:
Green Level:
Expects to receive both financial and personal rewards in exchange for applying his or her best energies to the job. Enjoys working. Trusts the organization to help career.
Yellow-Green Level:
Usually expects to receive both financial and personal rewards in exchange for solid and consistent effort on the job. Enjoys most work activities and is willing to put in extra effort when warranted or requested.
Yellow Level:
Dedicates a medium level of effort to his or her work but is willing to allocate extra effort when asked and rewarded for doing so. Typically neutral regarding job satisfaction and/or enjoyment. Usually trusts management, but can become skeptical when prompted by unexpected circumstances.
Red-Yellow Level:
Sees work as a secondary activity. Most of the time does not enjoy working and often applies average effort at best. Tends to be skeptical regarding management decisions and company policies.
Red Level:
Sees work as a secondary activity to be mostly endured. Dedicates just enough energy to get by. May distrust the organization to assist with his or her career.
Interview Question:
Question:
How do you feel having a regular job and going to work? Is it something you enjoy? Why or why not?
High Anchor:
Views work as a means of income only. Does not enjoy. Does not care about professional reputation.
Medium Anchor:
Likes work but doesn't truly enjoy it. Balances priority and energy with other obligations.
Low Anchor:
Considers work a key priority in life. Enjoys working and always applies best energy. Takes pride in work reputation.
Scoring Parameters and scoring method:
First, sum the individual item scores for all items in this scale.
Next, use this loookup table to convert the total for each scale to a 0-100 scale:
Total,0-100 Score
11,5,
13,10,
17,15,
20,20,
21,35,
22,45,
23,50,
24,55,
25,60,
26,65,
27,70,
29,75,
30,80,
31,85,
32,90,
34,95,
36,100
Note: A lookup table is not required. We can also use total percent correct, for instance. Please contact us on how you want to score the competency. This is just an example.
Score Cutoffs:
Green: 80 - 100
Yellow-Green: 60-80
Yellow: 40 - 60
Red-Yellow: 20 - 40
Red: 0 - 20
It's not trivial, but it's doable, particularly for a large client!
If bundled, the new competencies can be calculated as part of the overall score and totally integrated into the test. However, to do this, we need the items, item scores, and a set of additional information:
- Competency Description
- Score Text - this is text that appears in the report and online system.
- Interview Question - for the report
- Score Parameters
- Score Cutoffs
Description:
For some people, work is a second-place activity. That is, given a decision to take either personal time or go to work, low scorers will choose time off. Low job priority could indicate a 9 to 5 mentality. Sample item: It is OK to take long lunches and breaks if you are underpaid.
Score Text:
Green Level:
Expects to receive both financial and personal rewards in exchange for applying his or her best energies to the job. Enjoys working. Trusts the organization to help career.
Yellow-Green Level:
Usually expects to receive both financial and personal rewards in exchange for solid and consistent effort on the job. Enjoys most work activities and is willing to put in extra effort when warranted or requested.
Yellow Level:
Dedicates a medium level of effort to his or her work but is willing to allocate extra effort when asked and rewarded for doing so. Typically neutral regarding job satisfaction and/or enjoyment. Usually trusts management, but can become skeptical when prompted by unexpected circumstances.
Red-Yellow Level:
Sees work as a secondary activity. Most of the time does not enjoy working and often applies average effort at best. Tends to be skeptical regarding management decisions and company policies.
Red Level:
Sees work as a secondary activity to be mostly endured. Dedicates just enough energy to get by. May distrust the organization to assist with his or her career.
Interview Question:
Question:
How do you feel having a regular job and going to work? Is it something you enjoy? Why or why not?
High Anchor:
Views work as a means of income only. Does not enjoy. Does not care about professional reputation.
Medium Anchor:
Likes work but doesn't truly enjoy it. Balances priority and energy with other obligations.
Low Anchor:
Considers work a key priority in life. Enjoys working and always applies best energy. Takes pride in work reputation.
Scoring Parameters and scoring method:
First, sum the individual item scores for all items in this scale.
Next, use this loookup table to convert the total for each scale to a 0-100 scale:
Total,0-100 Score
11,5,
13,10,
17,15,
20,20,
21,35,
22,45,
23,50,
24,55,
25,60,
26,65,
27,70,
29,75,
30,80,
31,85,
32,90,
34,95,
36,100
Note: A lookup table is not required. We can also use total percent correct, for instance. Please contact us on how you want to score the competency. This is just an example.
Score Cutoffs:
Green: 80 - 100
Yellow-Green: 60-80
Yellow: 40 - 60
Red-Yellow: 20 - 40
Red: 0 - 20
How We Calculate Scores
NOTE: SCORING METHOD CHANGED 7/1/2018. THE FOLLOWING ENTRY IS NO LONGER VALID. PLEASE REFER TO THE NEW METHOD BLOG ENTRY.
This post is divided into two sections,
- Individual Competency Score Calculations
- Overall Score Calculation
Keep in mind that Overall Score is calculated to be essentially a Normal Distribution with a mean of 50 and a standard deviation of 21. It is not a percentile.
To get the overall score, we must first obtain scores for each individual competency. We then combine these scores in order to arrive at the overall score.
Step 1: Individual Competency Score Calculation
- For cognitive modules, we calculate the score as the percent of available points the test-taker achieved.
- For AIMS, we sum the points from the indiv items (likert scale), apply a lookup table to convert to 0-100.
- For Biodata, we use the average points per item and then apply a lookup table to convert to 0-100.
- For skills/knowledge tests, we calculate the percent of available points.
Overall Score Calculation
The overall raw score is the weighted average of the individual competency scores. The overall final score is a standardized, then transformed version of the raw score.
The basic steps are:
- Determine the score value to use for each competency. In many cases it's the competency score, but not all.
- Set the weights for each competency using a standard formula.
- Adjust weights if needed.
- Apply ONET to the weights.
- Calculate raw score as weighted average of individual competency scores
- Standardize the raw score to a z score (mean=0, standard deviation=1)
- Transform the z score to a scale with a mean of 50 and a standard deviation of 21.06
Step 1: Determine the score value to use for each competency.
Some competencies, such as behavioral history and some attitudes, interests, and motivations are set to use the assigned color category rather than the numeric value for that competency. This is because the more desirable regions of the score scale may be in different parts of the scale. For instance, Develops Relationships is best in the middle of the scale rather than at the high end. Or, Past Behaviors may be skewed highly to the high end of the scale.
Step 2. Set standard weights for each competency such that:
- total weights for all Cognitive Ability competencies = 1
- total weights for all Skills/Knowledge competencies = 0.8
- total weights for all AIMS competencies = 0.7
- Total weights for all Biodata competencies = 0.4
For individual competencies within a category, the weights are spread evenly (i.e. 1/n each).
Step 2. For some jobs, we add some weight to specific competencies. For instance, we increase the weight on the Expressive AIMS factor for sales people. This weight will override the weight determined in step 1.
Step 3. Next, apply ONET importance ratings to each competency. Since not all competencies have an onet importance value, those that don't are assigned an onet importance of 4.2 (80% on a 1-5 ONET importance scale). This is to prevent these from getting extra weight just because they are not in ONET. Most ONET weights are below this value and the average is about 2.5. The ONET value is converted to a fraction and applied to the weight from steps 1 and 2. Weight fm step 1 * ONET as fraction = final weight.
Step 4. The overall raw score is the weighted average of the individual competency scores.
Step 5. An overall z score is calculated using a mean/std derived from existing data (if there are over 100 test events in dbms) for that test or "all tests" if there is not enough data. The derived mean measured on about 100 events after changing the above is 54.6, std=9.15 These are the values I'm using.
Step 6. The overall z-score is transformed to a score using a mean of 50 and a std of 21.06. These values are the values used for a Normal Curve Equivalent Scale (NCE) used in various achievement tests. If the score is outside the range of 0-100, it is set to 0 or 100 as appropriate.
Notes:
1. Key values are the mean and standard deviation of the overall raw score (the weighted average), which is used to create the z score. These values are based on test results for this test. If there are not enough completed results for a given test, overall mean and standard deviation values are used.
2. Sometimes the overall score will look higher than it should be, or lower than it should be. This is because the overall score is not just a raw weighted average, which most people assume. It's a reflection of the number of standard deviations from the mean that the person scored. So, if their overall weighted average is 40, but the average overall weighted average is 80 and the standard deviation is 20, then they are two standard deviations below the mean. So when converted to NCE (mean 50 and std 21.06), they will have an overall score of 50 - 2*21.06 = 7.88. So, the overall score will say 8.
Note: For information on the Normal Curve Equivalent Scale (NCE) go to:
http://en.wikipedia.org/wiki/Normal_curve_equivalent
Monday, January 19, 2015
Testing Science Resources
We will use this page to hold science and psychometrics resources.
Short Version: HR Avatar Test Development Summary
Testing Science Page on HR Avatar Website
Technical Manual
Test User Manual
Short Version: HR Avatar Test Development Summary
Testing Science Page on HR Avatar Website
Technical Manual
Test User Manual
Partner Resources & Link Reference
News
https://www.hravatar.com/news
Blog
https://www.hravatar.com/blog
Social Links
Twitter:
@hravatarmike
@HR_Avatar
LinkedIn:
https://www.linkedin.com/company/hr-avatar?trk=company_logo
https://www.facebook.com/HRAvatar
CD-ROM Demos
Cognitive Face-To-Face Demo
Download this and unzip to a CD ROM or Memory stick for a demo. Load and click on player.exe Works on PCs only.
Print Logos
300 DPI Logo Source File
Web Logos
White Papers
The Fastest Way to Improve Hiring
US Merit Systems Protection Board White Paper on Simulations for Hiring Selection
Mike Russiello's Book, Better hires, Fewer Fires On Amazon
Software Integration Documents
HR Avatar Technology Platform
HR Avatar Web Services Integration Guide
ClicFlic IMO API
Partner Administration Basics
Our intent is for partners to be able to manage all aspects of their distribution partnership with HR Avatar from the Partner Administration Menu.
Always Feel Free to Contact Us
Shoa Appelman: (o) (703) 938-1615 (m) 703-966-2080, shoa@hravatar.com
Mike Russiello: (o) (703) 938-1615 (m)571-213-5677, mike@hravatar.com
Or use our online form:
https://www.hravatar.com/contact
Accessing the Partner Administration Menu
HR Avatar staff will create the Source Account for each partner. The Partner then can log onto the Source Account. All users designated as Account Administrators" in the Source Account will have an option in their pull-down menu entitled "Partner Admin." Clicking on this option will bring up the Partner Administration Menu.
The Menu provides access to all of the functions below:
Creating New Accounts
From the menu, click on "Create New Account." This will bring up a form for all necessary information needed to create the account. Note that the Administrative User should be a member of the Partner Organization.
Accounts created using this form do not require activation in the way accounts that created off the HR Avatar website directly do.
The Account Administrative User
When you create a new account, you must specify an administrative user for this account.
The Administrative User should be a member of the Partner Organization. Once this user is created, you can log in as this user and create additional users within the newly-created account, such as for your customers to log on and view results. All users you create will have the same rights as normal users (you can choose their rights when you create them), except they cannot purchase credits for their account for Partner-Managed accounts. Instead, they must contact you, the partner, to obtain credits in their account.
Logging Onto A Partner-Managed or Customer-Managed Account
Once you've created an account, you can log onto that account from the Partner Administration Menu by selecting Partner-Managed Accounts or Customer-Managed Accounts, then clicking the login link for the relevant account.This will log you out of the source account and log you into the specific partner-managed or customer-managed account. From here you can perform all functions that any account administrator can perform - such as adding new users, creating sub-organizations, or setting defaults for test result distribution.
Additional Account Users
Once you've created an account, log onto that account from the Partner Administration Menu using the instructions above. Then go to Account-Settings -> Users in the pull-down menu. From here you can create user logins for your customers (if you want them to have logins). Note that you can define the rights of any user as a Read Only (can view test results only), Basic User (can authorize test events and review results), or Account Administrator (can perform all functions except purchase credits).
Opening the Account Info Page
For the Source Account click on View Source Account from the Partner Administration Menu.
For Partner-Managed or Customer-Managed accounts, you can select the appropriate "View Accounts" link from the Partner Administration Menu and click on the name of the account you want to view. When the Account Info Page opens, all relevant information for that account should be presented.
For all account types, you can make various configuration changes from this screen.
For partner-managed accounts, you can also designate or undesignate accounts as "Demo" accounts. You can also add credits to the account from this screen for source and partner-managed accounts.
Adding Credits to an Account
For either the source account or any partner-managed account, you can add credits as follows:
Step 1: Open the account info page for the account you want to add credits to.
Step 2: Using the form at the bottom of the page, enter the number of credits and the credit expiration date. Note that this date cannot be longer than one year in the future.
Designating an Account as a Demo Account
Partner-managed and source accounts can be designated as demo accounts. This means that all credits added to the account are denoted as demo credits and are not charged to the partner. This should only be used for demonstration tests that in which the customer is not asked to pay for the tests.
To designate an account as a demo account, click the checkbox entitled: Demo Account. From that point until the box is unchecked, credits placed in this account will be demo credits.
Converting an account from Demo to Paid
When the account is ready to leave demo status, you can uncheck the Demo Account checkbox and save. This will cause the system to make all demo credits expire within 24 hours.
Note that once an account has been in and out of demo status once, it cannot be redesignated as a demo account without contacting HR Avatar.
Reporting Test Activity for Each Account
The account info page has data regarding the activity level of each account. However, for aggregate information across all your accounts, you can use the Affiliate Activity Report from the Partner Administration Menu. Simply enter the date range and create the report. Unless there are special circumstances, this is the same report that will be used to calculate the invoice (or check) that is sent you at the end of each accounting period.
Always Feel Free to Contact Us
Shoa Appelman: (o) (703) 938-1615 (m) 703-966-2080, shoa@hravatar.com
Mike Russiello: (o) (703) 938-1615 (m)571-213-5677, mike@hravatar.com
Or use our online form:
https://www.hravatar.com/contact
Accessing the Partner Administration Menu
HR Avatar staff will create the Source Account for each partner. The Partner then can log onto the Source Account. All users designated as Account Administrators" in the Source Account will have an option in their pull-down menu entitled "Partner Admin." Clicking on this option will bring up the Partner Administration Menu.
The Menu provides access to all of the functions below:
Creating New Accounts
From the menu, click on "Create New Account." This will bring up a form for all necessary information needed to create the account. Note that the Administrative User should be a member of the Partner Organization.
Accounts created using this form do not require activation in the way accounts that created off the HR Avatar website directly do.
The Account Administrative User
When you create a new account, you must specify an administrative user for this account.
The Administrative User should be a member of the Partner Organization. Once this user is created, you can log in as this user and create additional users within the newly-created account, such as for your customers to log on and view results. All users you create will have the same rights as normal users (you can choose their rights when you create them), except they cannot purchase credits for their account for Partner-Managed accounts. Instead, they must contact you, the partner, to obtain credits in their account.
Logging Onto A Partner-Managed or Customer-Managed Account
Once you've created an account, you can log onto that account from the Partner Administration Menu by selecting Partner-Managed Accounts or Customer-Managed Accounts, then clicking the login link for the relevant account.This will log you out of the source account and log you into the specific partner-managed or customer-managed account. From here you can perform all functions that any account administrator can perform - such as adding new users, creating sub-organizations, or setting defaults for test result distribution.
Additional Account Users
Once you've created an account, log onto that account from the Partner Administration Menu using the instructions above. Then go to Account-Settings -> Users in the pull-down menu. From here you can create user logins for your customers (if you want them to have logins). Note that you can define the rights of any user as a Read Only (can view test results only), Basic User (can authorize test events and review results), or Account Administrator (can perform all functions except purchase credits).
Opening the Account Info Page
For the Source Account click on View Source Account from the Partner Administration Menu.
For Partner-Managed or Customer-Managed accounts, you can select the appropriate "View Accounts" link from the Partner Administration Menu and click on the name of the account you want to view. When the Account Info Page opens, all relevant information for that account should be presented.
For all account types, you can make various configuration changes from this screen.
For partner-managed accounts, you can also designate or undesignate accounts as "Demo" accounts. You can also add credits to the account from this screen for source and partner-managed accounts.
Adding Credits to an Account
For either the source account or any partner-managed account, you can add credits as follows:
Step 1: Open the account info page for the account you want to add credits to.
Step 2: Using the form at the bottom of the page, enter the number of credits and the credit expiration date. Note that this date cannot be longer than one year in the future.
Designating an Account as a Demo Account
Partner-managed and source accounts can be designated as demo accounts. This means that all credits added to the account are denoted as demo credits and are not charged to the partner. This should only be used for demonstration tests that in which the customer is not asked to pay for the tests.
To designate an account as a demo account, click the checkbox entitled: Demo Account. From that point until the box is unchecked, credits placed in this account will be demo credits.
Converting an account from Demo to Paid
When the account is ready to leave demo status, you can uncheck the Demo Account checkbox and save. This will cause the system to make all demo credits expire within 24 hours.
Note that once an account has been in and out of demo status once, it cannot be redesignated as a demo account without contacting HR Avatar.
Reporting Test Activity for Each Account
The account info page has data regarding the activity level of each account. However, for aggregate information across all your accounts, you can use the Affiliate Activity Report from the Partner Administration Menu. Simply enter the date range and create the report. Unless there are special circumstances, this is the same report that will be used to calculate the invoice (or check) that is sent you at the end of each accounting period.
Sunday, January 18, 2015
Unlimited vs Credit Accounts
Remember that there are three types of accounts:
1. Source Account - this is the main Partner Account.
2. Partner-Managed Account - These are accounts used by your customers that the Partner administers the account and adds all credits into the account. The customer can still log on and administer tests or review results. However, this is controlled by the partner. We anticipate that most accounts will be partner-managed.
3. Customer-Managed Accounts - These are accounts where your customer administers the account and add credits by themselves. HR Avatar reimburses the partner for purchases made by each customer.
A Partner-Managed account or Source account can be designated either as Unlimited or as a Credit-based.
Unlimited Accounts - are just that - unlimited. There are no credits used for an unlimited account. When you designate an account as unlimited, you must also indicate an expiration date. When the date is reached, the account is treated by the testing system as a credit-based account. Partners can log on to extend the expiration date if needed.
Remember that in most cases, your agreement requires HR Avatar permission before designating an account as unlimited.
You can also set a max number of test events for an unlimited account. If you set the Max Test Events value to 0, the account will not have any limit.
Credit-Based Accounts - use credits. Partner-Managed accounts require that the partner log on and add credits to an account. When all credits are used up the testing system will not allow any further testing until new credits are added.
A credit-based account can be designated as Paid or Demo. Credits added to 'demo' partner-managed accounts are not charged to the partner, since they are assumed to be for demonstration purposes only.
All customer-managed accounts are credit-based.
1. Source Account - this is the main Partner Account.
2. Partner-Managed Account - These are accounts used by your customers that the Partner administers the account and adds all credits into the account. The customer can still log on and administer tests or review results. However, this is controlled by the partner. We anticipate that most accounts will be partner-managed.
3. Customer-Managed Accounts - These are accounts where your customer administers the account and add credits by themselves. HR Avatar reimburses the partner for purchases made by each customer.
A Partner-Managed account or Source account can be designated either as Unlimited or as a Credit-based.
Unlimited Accounts - are just that - unlimited. There are no credits used for an unlimited account. When you designate an account as unlimited, you must also indicate an expiration date. When the date is reached, the account is treated by the testing system as a credit-based account. Partners can log on to extend the expiration date if needed.
Remember that in most cases, your agreement requires HR Avatar permission before designating an account as unlimited.
You can also set a max number of test events for an unlimited account. If you set the Max Test Events value to 0, the account will not have any limit.
Credit-Based Accounts - use credits. Partner-Managed accounts require that the partner log on and add credits to an account. When all credits are used up the testing system will not allow any further testing until new credits are added.
A credit-based account can be designated as Paid or Demo. Credits added to 'demo' partner-managed accounts are not charged to the partner, since they are assumed to be for demonstration purposes only.
All customer-managed accounts are credit-based.
Friday, January 16, 2015
Types of Accounts
There are three types of accounts used by partners:
1. Source Account - this is the main account for the partner. All partner accounts can be managed by logging onto the source account with an account administrator logon. Source accounts can be designated as demo accounts and used only for delivering tests for demonstration purposes.
2. Partner-Managed Account - This is a customer account that is created and managed by the partner. Only the partner can add credits or change the demo status of a partner-managed account. The partner "sells" credits to the customer and HR Avatar bills the partner for its share of the revenue on a periodic basis.
Partner managed accounts can be designated as demo accounts and used only for delivering tests for demonstration purposes. Credits must still be added to demo accounts, but the credits are not 'charged' to the partner. At some point in time the account can be designated as a non-demo account when the customer begins to pay for credits. A partner can designate an account as a demo account only once, for any duration. When it is converted back to a non-demo account, it cannot be re-converted to a demo account without help from the HR Avatar staff.
Note also that partner-managed accounts can also be designated as 'unlimited' accounts. These types of accounts ignore credits altogether. However, you cannot designate an account as both 'Demo' and Unlimited. The difference is that 'Demo' accounts must still add credits, even though the partner is not charged for credits. Unlimited accounts are totally unlimited and do not require credits.
For accounting purposes, you should use demo accounts when giving out free tests to prospective customers. Unlimited accounts are monitored very closely and should not be used for demo purposes.
3. Customer-Managed Account - This is an account created by a customer within the designated region of the partner in which all accounts are automatically associated with the partner. The Customer purchases credits directly via the website and HR Avatar reimburses the partner for the partner's share of the revenue from each purchase.
How Accounts Are Created:
HR Avatar staff will create the Source Account for a partner. The Partner then can log onto the Source Account. All account administrators in the Source Account will have an option in their pull-down menu entitled "Partner Admin." Clicking on this option will bring up the Partner Administration Menu. From this menu, you can create additional Partner-Managed accounts by clicking on "Create an New Account."
Customer-Managed Accounts are usually created by the customer themselves, simply by registering on www.hravatar.com for their own account. Typically, the system recognizes their country by IP address and makes them customer-managed account automatically. However, if you would like to create a new customer-managed account, you can do this by clicking on "Create an New Account" from the Partner Administration menu and selecting Customer-Managed from the Account Type entry.
Most accounts will be Partner-Managed.
1. Source Account - this is the main account for the partner. All partner accounts can be managed by logging onto the source account with an account administrator logon. Source accounts can be designated as demo accounts and used only for delivering tests for demonstration purposes.
2. Partner-Managed Account - This is a customer account that is created and managed by the partner. Only the partner can add credits or change the demo status of a partner-managed account. The partner "sells" credits to the customer and HR Avatar bills the partner for its share of the revenue on a periodic basis.
Partner managed accounts can be designated as demo accounts and used only for delivering tests for demonstration purposes. Credits must still be added to demo accounts, but the credits are not 'charged' to the partner. At some point in time the account can be designated as a non-demo account when the customer begins to pay for credits. A partner can designate an account as a demo account only once, for any duration. When it is converted back to a non-demo account, it cannot be re-converted to a demo account without help from the HR Avatar staff.
Note also that partner-managed accounts can also be designated as 'unlimited' accounts. These types of accounts ignore credits altogether. However, you cannot designate an account as both 'Demo' and Unlimited. The difference is that 'Demo' accounts must still add credits, even though the partner is not charged for credits. Unlimited accounts are totally unlimited and do not require credits.
For accounting purposes, you should use demo accounts when giving out free tests to prospective customers. Unlimited accounts are monitored very closely and should not be used for demo purposes.
3. Customer-Managed Account - This is an account created by a customer within the designated region of the partner in which all accounts are automatically associated with the partner. The Customer purchases credits directly via the website and HR Avatar reimburses the partner for the partner's share of the revenue from each purchase.
How Accounts Are Created:
HR Avatar staff will create the Source Account for a partner. The Partner then can log onto the Source Account. All account administrators in the Source Account will have an option in their pull-down menu entitled "Partner Admin." Clicking on this option will bring up the Partner Administration Menu. From this menu, you can create additional Partner-Managed accounts by clicking on "Create an New Account."
Customer-Managed Accounts are usually created by the customer themselves, simply by registering on www.hravatar.com for their own account. Typically, the system recognizes their country by IP address and makes them customer-managed account automatically. However, if you would like to create a new customer-managed account, you can do this by clicking on "Create an New Account" from the Partner Administration menu and selecting Customer-Managed from the Account Type entry.
Most accounts will be Partner-Managed.
Subscribe to:
Posts (Atom)