What's that noise? It's a delicate, almost musical sound, like a light falling of rain, only slightly more metallic. Could it be the sound of pennies dropping in boardrooms across the AEC industry?
For too long, CAD has been seen as a bit of a pariah within many architectural & engineering firms. The poor cousin; the whinging child in the corner. CAD 'Managers' are dismissed as slightly odd creatures, always working late (unpaid), often crawling around under desks with a pen or mini-screwdriver firmly planted behind one ear, slightly dishevelled, with odd socks and bits of network cable dangling around their necks like ill-conceived costume jewellery. And I use the term 'Manager' lightly, because how many firms actually extend their CAD leaders the courtesy of a senior management position? And yet, they are often the unsung heroes (or heroines) of their firm. Try being without them for a week and see what anarchy ensues!
But lately, it would seem, perhaps as a necessity brought about by the decline in our collective economic fortunes, that things are starting to change. There is a shift in the breeze and senior AEC management are waking up to the idea that CAD (and more likely BIM) might actually be a bit more relevant than they first thought. And furthermore, it might even be the very thing that keeps them in business long enough to enjoy the inevitable upswing in the economy.
Because what is the main deliverable of an architectural firm? Designing buildings, you say? No! The chief deliverable for an architect - it's 'product', if you will - is working drawings, production information, models! Ditto for engineers. And what medium do they use to create said products? That's right - CAD (or latterly BIM) software.
So the firms who have had their lightbulb moment, their penny dropping experience if you will, are now involving CAD & BIM not just at the periphery of their offices - but at the very heart of their enterprise. Not just as another costly, balance-sheet-sucking IT resource, but as a business critical investment that can be the difference between bids won and lost.
Take, for example, clients such as ASDA or the Army Corps, who are now specifying BIM tools as part of their project briefs. Without the ability to demonstrate proficiency in these tools, firms won't even make the long-list, let alone the short-list for future work.
CADsmart finds itself at the heart of this technology debate. Consider the following testimonial from one of our large customers (400+ users, multiple offices, mix of Autodesk & Bentley tools);
“CADsmart is fundamental in the assessment of CAD abilities for existing and potential staff. It provides increased business awareness of current skill-sets, records valuable cross office KPIs and enables management to better resource and deliver training within budgets. It improves business efficiency, productivity and profitability, enabling staff to gain skills and confidence to the mutual benefit of all concerned. It enables line managers to better assess their teams’ skills and contributes to annual staff reviews. The business is able to recognise and reward the best staff whilst encouraging them to become mentors to others. It fosters a healthy competitive environment amongst staff and establishes a desire for self improvement. CADsmart drives capital investment in CAD/BIM technologies and has become an important business management, analysis and ROI tool.”
Not surprisingly, this firm has posted some of the highest test scores amongst our large customers. It has a well managed CAD & BIM resource, which sits at the heart of the business - and which has the attention of senior management.
Could this be the model for all firms to follow, as we head down the road to BIM? How seriously does your leadership take their commitment to CAD & BIM?
Rory
Showing posts with label CAD. Show all posts
Showing posts with label CAD. Show all posts
Friday, 20 February 2009
Monday, 2 February 2009
Recession - a good time for CAD & BIM training?
As the recessionary typhoon continues unabated, drawing us all into its swirling centre, the question of staff development crops up time and again.
It seems as if everyone is looking at their budgets to see how they can be reduced. In most firms, financial controllers are looking at training budgets and assessing the benefits of immediately using some of the funds elsewhere or removing them all together! It is a difficult situation and one that presents CAD & HR Managers, and others responsible for training staff, with a serious challenge. How can they reach their targets - and keep projects on track - if their workforce is under-developed?
In Britain, the Learning Skills Council is encouraging businesses to take advantage of government funding for training, which may increase motivation and productivity; "Those (businesses) that make the right investment in training and development will be in a much better place when we come out of the recession. Training is one of the cornerstones of any successful business today, as it makes the organisation adaptable to change and therefore more productive. The last thing you should do is cut your training; now is not the time to lose employees to competition."
Research from the Cranfield School of Management shows that it is cheaper and more effective to 'grow your own' employees rather than shop around for talent. Cranfield's 'Nurturing Talent' report, which canvassed the views of 1,200 training and recruitment decision makers, found that three quarters of employers would rather nurture and develop talent in-house than recruit externally.
Almost half of small and medium sized businesses that develop in-house talent noted a cost saving, 20% recorded improved staff motivation and 45% increased employee retention. The report concludes; "For employers, the nurturing talent concept means managing and developing employees to achieve business goals. This could include training, employee coaching, staff mentoring and job enrichment to stretch employees with new tasks."
A leading US business performance coach states simply; "Train intensively in tough times... and less so in good times. In good times you want employees to be working hard, setting new performance records and serving customers; taking advantage of the good times. Prepare for the good times with intense internal training in tough times - you won't get a better opportunity!"
From our experience at CADsmart, we see AEC firms falling into two distinct categories; those who 'get it' and those who don't. The ones who do understand these principles have a rolling, continuous improvement environment for their CAD & BIM teams, including regular assessment, targeted training, buddying & mentoring schemes - and a clear path from basic skills through to advanced CAD & BIM applications for projects.
The ones who don't, seem to be taking more of a head-in-the-sand approach to their planning. They cut skills assessment from their priority list, along with training & development programs. Their rationale? We're not hiring, so we don't need skills evaluations. These same firms, when it comes to the unfortunate position of letting people go, have no real clue who their best - most productive - users are! They slice 5%, 10%, 20% from their payrolls - losing super-users alongside inefficient ones - without thought for the medium-term consequences!
Sensible business leaders are keen to keep staff motivated at all times. Good leadership is critical during times of fear, uncertainty and pessimism such as these. Companies need leaders who can manage their emotions, stay positive and show resolve. Providing development through internal mentoring, using senior users to mentor less experienced colleagues, can be very effective especially if salaries and promotion are temporarily pegged. The best CAD users will stay only if they are receiving development.
So, when faced with that budget challenge from your FD or CFO, make the case as strongly as you can that your training resources should take priority - and your training programs should be tailored and targeted. If you can achieve this - you can guarantee it will be effective.
Rory
It seems as if everyone is looking at their budgets to see how they can be reduced. In most firms, financial controllers are looking at training budgets and assessing the benefits of immediately using some of the funds elsewhere or removing them all together! It is a difficult situation and one that presents CAD & HR Managers, and others responsible for training staff, with a serious challenge. How can they reach their targets - and keep projects on track - if their workforce is under-developed?
In Britain, the Learning Skills Council is encouraging businesses to take advantage of government funding for training, which may increase motivation and productivity; "Those (businesses) that make the right investment in training and development will be in a much better place when we come out of the recession. Training is one of the cornerstones of any successful business today, as it makes the organisation adaptable to change and therefore more productive. The last thing you should do is cut your training; now is not the time to lose employees to competition."
Research from the Cranfield School of Management shows that it is cheaper and more effective to 'grow your own' employees rather than shop around for talent. Cranfield's 'Nurturing Talent' report, which canvassed the views of 1,200 training and recruitment decision makers, found that three quarters of employers would rather nurture and develop talent in-house than recruit externally.
Almost half of small and medium sized businesses that develop in-house talent noted a cost saving, 20% recorded improved staff motivation and 45% increased employee retention. The report concludes; "For employers, the nurturing talent concept means managing and developing employees to achieve business goals. This could include training, employee coaching, staff mentoring and job enrichment to stretch employees with new tasks."
A leading US business performance coach states simply; "Train intensively in tough times... and less so in good times. In good times you want employees to be working hard, setting new performance records and serving customers; taking advantage of the good times. Prepare for the good times with intense internal training in tough times - you won't get a better opportunity!"
From our experience at CADsmart, we see AEC firms falling into two distinct categories; those who 'get it' and those who don't. The ones who do understand these principles have a rolling, continuous improvement environment for their CAD & BIM teams, including regular assessment, targeted training, buddying & mentoring schemes - and a clear path from basic skills through to advanced CAD & BIM applications for projects.
The ones who don't, seem to be taking more of a head-in-the-sand approach to their planning. They cut skills assessment from their priority list, along with training & development programs. Their rationale? We're not hiring, so we don't need skills evaluations. These same firms, when it comes to the unfortunate position of letting people go, have no real clue who their best - most productive - users are! They slice 5%, 10%, 20% from their payrolls - losing super-users alongside inefficient ones - without thought for the medium-term consequences!
Sensible business leaders are keen to keep staff motivated at all times. Good leadership is critical during times of fear, uncertainty and pessimism such as these. Companies need leaders who can manage their emotions, stay positive and show resolve. Providing development through internal mentoring, using senior users to mentor less experienced colleagues, can be very effective especially if salaries and promotion are temporarily pegged. The best CAD users will stay only if they are receiving development.
So, when faced with that budget challenge from your FD or CFO, make the case as strongly as you can that your training resources should take priority - and your training programs should be tailored and targeted. If you can achieve this - you can guarantee it will be effective.
Rory
Monday, 5 January 2009
Egon Brunswick's 'Lens Model of Perception'
An obscure title for a CAD blog post, you might think!
In 1937 Professor Egon Brunswick conceived of a model of social perception, called the "Lens Model". The essence of the Lens is this, "What we think we see in others will determine, for the most part, how we will treat them and respond to them. The accuracy of what we think we see will dictate the appropriateness of the behaviour we utilize or the actions we take and the resulting productivity of the relationship."
As humans, we demonstrate the same behaviour in all aspects of life, not just in respect of how we see other people, per se, but in just about everything. We tend to develop pre-conceived ideas about all manner of topics (rightly or wrongly) and form opinions on them instantly when the subject is raised. In effect, we view everything in life through our own personal 'lens', based on a combination our experiences up to that point.
OK, enough of the psychology 101 - which brings me to my point! I had a good chuckle last week upon reading a post on Autodesk's discussion forum. The poster asked, 'Has anyone used this (CADsmart software)? If so, can you give me your opinion on it?'
A perfectly valid question, you might point out - which of course it is. The replies, however, demonstrated some interesting facets about the personalities of people who frequent these boards. Of the replies posted, only one had actually read the question fully - 'has anyone used the software?' i.e. can they give me an opinion based on the facts, not personal opinion based on heresay, ego, etc. etc. This person had used the software, and replied with an honest, positive account of their experience.
The remainder simply seemed to want to vent their collective CAD spleen on the virtues (or lack thereof) of testing CAD skills in general, based purely on their own personal biases on the subject (their 'lens' if you will). Not surprisingly, none had anything particularly positive to relay, but were nevertheless happy to share their (ill-researched) opinions with the world - don't they always?!
The most amusing post stated, in apparent horror, '$500 for a 10 minute test ? WOW ! I can't imagine anyone spending thousands of dollars to find out which candidate performed the best on a ten minute test. The money might be better spent hiring an private investigator to check out their resume!'
Which is funny up to a point, but demonstrates only a partial grasp of the facts and a huge (incorrect) assumption, which they were happy to splash all over the discussion boards.
Oh, and for the record, the $497 subscription for our Xpress software, which does indeed take 10-20 minutes to complete, offers unlimited assessments for a whole year - slightly better value, wouldn't you think? And for a more detailed assessment, including training needs analysis on a user's weak areas, the Premier service typically costs $10 - $60 per user per year, depending on the firm size.
Happy New Year!
Rory
In 1937 Professor Egon Brunswick conceived of a model of social perception, called the "Lens Model". The essence of the Lens is this, "What we think we see in others will determine, for the most part, how we will treat them and respond to them. The accuracy of what we think we see will dictate the appropriateness of the behaviour we utilize or the actions we take and the resulting productivity of the relationship."
As humans, we demonstrate the same behaviour in all aspects of life, not just in respect of how we see other people, per se, but in just about everything. We tend to develop pre-conceived ideas about all manner of topics (rightly or wrongly) and form opinions on them instantly when the subject is raised. In effect, we view everything in life through our own personal 'lens', based on a combination our experiences up to that point.
OK, enough of the psychology 101 - which brings me to my point! I had a good chuckle last week upon reading a post on Autodesk's discussion forum. The poster asked, 'Has anyone used this (CADsmart software)? If so, can you give me your opinion on it?'
A perfectly valid question, you might point out - which of course it is. The replies, however, demonstrated some interesting facets about the personalities of people who frequent these boards. Of the replies posted, only one had actually read the question fully - 'has anyone used the software?' i.e. can they give me an opinion based on the facts, not personal opinion based on heresay, ego, etc. etc. This person had used the software, and replied with an honest, positive account of their experience.
The remainder simply seemed to want to vent their collective CAD spleen on the virtues (or lack thereof) of testing CAD skills in general, based purely on their own personal biases on the subject (their 'lens' if you will). Not surprisingly, none had anything particularly positive to relay, but were nevertheless happy to share their (ill-researched) opinions with the world - don't they always?!
The most amusing post stated, in apparent horror, '$500 for a 10 minute test ? WOW ! I can't imagine anyone spending thousands of dollars to find out which candidate performed the best on a ten minute test. The money might be better spent hiring an private investigator to check out their resume!'
Which is funny up to a point, but demonstrates only a partial grasp of the facts and a huge (incorrect) assumption, which they were happy to splash all over the discussion boards.
Oh, and for the record, the $497 subscription for our Xpress software, which does indeed take 10-20 minutes to complete, offers unlimited assessments for a whole year - slightly better value, wouldn't you think? And for a more detailed assessment, including training needs analysis on a user's weak areas, the Premier service typically costs $10 - $60 per user per year, depending on the firm size.
Happy New Year!
Rory
Tuesday, 30 December 2008
Benchmarking - what's it all about?
Benchmarking is the process of comparing the cost, time or quality of what one organization does against what another organization does. The result is often a business case for making changes in order to make improvements. Also referred to as "best practice benchmarking" or "process benchmarking", it is a process used in management where organizations evaluate various aspects of their processes in relation to best practice, usually within their own sector. This then allows organizations to develop plans on how to make improvements or adopt best practice, with the aim of increasing performance.
The most prominent methodology is the 12 stage approach by Robert Camp (who literally wrote the book on benchmarking in 1989). It consists of;
1. Select subject 2. Define the process 3. Identify potential partners 4. Identify data sources 5. Collect data and select partners 6. Determine the gap 7. Establish process differences 8. Target future performance 9. Communicate 10. Adjust goal 11. Implement 12. Review / recalibrate.
Benchmarking can take various guises:
Process benchmarking - a firm focuses its observation and investigation of business processes with a goal of identifying and observing the best practices from one or more benchmark firms. Activity analysis will be required where the objective is to benchmark cost and efficiency.
Financial benchmarking - a company performs a financial analysis and compares the results in an effort to assess overall competitiveness.
Performance benchmarking - allows a firm to assess their competitive position by comparing products and services with those of target firms.
Product benchmarking - the process of designing new products or upgrades to current ones. This process can sometimes involve reverse engineering competitors’ products to find strengths and weaknesses.
Strategic benchmarking - involves observing how others compete. This type is usually not industry specific, meaning it is best to look at other industries.
Functional benchmarking - a company will focus its benchmarking on a single function in order to improve the operation of that particular function, i.e. Human Resources, Finance and ICT.
Internal benchmarking - involves benchmarking businesses or operations from within the same organisation (e.g. business units in different countries).
External benchmarking - analysing outside organisations that are known to be best in class provides opportunities of learning from those who are at the ‘leading edge’.
International benchmarking - best practitioners are identified and analysed elsewhere in the world; globalisation and advances in information technology are increasing opportunities for international projects.
Process benchmarking - a firm focuses its observation and investigation of business processes with a goal of identifying and observing the best practices from one or more benchmark firms. Activity analysis will be required where the objective is to benchmark cost and efficiency.
Financial benchmarking - a company performs a financial analysis and compares the results in an effort to assess overall competitiveness.
Performance benchmarking - allows a firm to assess their competitive position by comparing products and services with those of target firms.
Product benchmarking - the process of designing new products or upgrades to current ones. This process can sometimes involve reverse engineering competitors’ products to find strengths and weaknesses.
Strategic benchmarking - involves observing how others compete. This type is usually not industry specific, meaning it is best to look at other industries.
Functional benchmarking - a company will focus its benchmarking on a single function in order to improve the operation of that particular function, i.e. Human Resources, Finance and ICT.
Internal benchmarking - involves benchmarking businesses or operations from within the same organisation (e.g. business units in different countries).
External benchmarking - analysing outside organisations that are known to be best in class provides opportunities of learning from those who are at the ‘leading edge’.
International benchmarking - best practitioners are identified and analysed elsewhere in the world; globalisation and advances in information technology are increasing opportunities for international projects.
Benchmarking involves four key steps:
1) Understand in detail existing business processes
2) Analyse the business processes of others
3) Compare own business performance with that of others
4) Implement the steps necessary to close the performance gap
Benchmarking should not be considered a one-off exercise. To be effective, it must become an ongoing, integral part of an ongoing improvement process with the goal of keeping abreast of ever-improving best practice.
Why Bother?
There are many benefits of benchmarking; the following list summarises the main ones:
· provides realistic and achievable targets
· prevents companies from being industry led
· challenges operational complacency
· encourages continuous improvement
· allows employees to visualise improvement which can be a strong motivator for change
· creates a sense of urgency for improvement
· confirms the belief that there is a need for change
· helps to identify weak areas and indicates what needs to be done to improve.
So how does all this apply to us - what are the benefits of benchmarking CAD performance?
· Gain visibility of core CAD skills
· Identify individuals’ strengths & weaknesses
· Implement better CAD training plans for staff
· Improve CAD recruitment processes
· Share performance data across an organisation
· Promote collaborative working between teams & offices
· Measure the performance of outsourcing or off-shoring partners
· Provide ‘best practice’ for CAD development
· Offer clearer staff inductions and more meaningful staff appraisals
· Enjoy better skills resourcing for projects
· Develop a continuous improvement process for CAD
· Save time and money on construction projects and offer best value for clients
· Gain visibility of core CAD skills
· Identify individuals’ strengths & weaknesses
· Implement better CAD training plans for staff
· Improve CAD recruitment processes
· Share performance data across an organisation
· Promote collaborative working between teams & offices
· Measure the performance of outsourcing or off-shoring partners
· Provide ‘best practice’ for CAD development
· Offer clearer staff inductions and more meaningful staff appraisals
· Enjoy better skills resourcing for projects
· Develop a continuous improvement process for CAD
· Save time and money on construction projects and offer best value for clients
In the end, it all comes down to better visibility of your teams and their real ability to use often complex technology and tools for maximum effect. If you can't measure it, you can't manage it.
Happy New Year!
Rory
Thursday, 11 December 2008
CAD skills testing job applicants in the USA - is it allowed?
Is it OK to test the basic CAD skills of prospective employees in the USA? This is a question which seems to come up quite frequently from our friends in the US. A common conundrum surrounding CAD skills assessments is whether it is legally allowed to screen the CAD ability of prospective employees and contract staff, before they join your team.
The short answer is yes, in our opinion, but only under the right circumstances. Let’s examine some of the evidence.
The most important issue underlying the use of pre-employment assessments is validity. The question we need to ask is this; ‘Is the test valid for this intended purpose; does it support the decisions that are going to be made?’
So what is 'validity'? Validity measures how appropriate a test is for a specific purpose. Simply put, a test may be considered valid for one use and invalid for another. Why do pre-employment tests need to be validated? In 1978, the Equal Employment Opportunities Commission (EEOC) created guidelines to ensure that the knowledge gained from testing is applied with impartiality to protect minority applicants from discriminatory employment procedures. What is the best method of validation? The EEOC guidelines do not state that one method is better than another; the method used must fit the needs of the business or organization.
There are three methods of validation set forth by the EEOC:
1) Criterion Validity: If data demonstrates that a test is significantly correlated with a vital measure of job performance, the test is said to demonstrate criterion validity. For example, if all the CAD users that scored highly on a selected test to measure CAD skills completed their projects accurately and on time, the test would demonstrate criterion validity.
2) Construct Validity: The term construct is a technical term for personality traits like intelligence and creativity. Construct validity is demonstrated if a test measures traits that have been found to influence successful performance of a job. A test that measures the interpersonal communication skills of a potential customer service representative would demonstrate construct validity.
3) Content Validity: This is demonstrated if the questions that make up an assessment are representative of content that is required to perform a particular activity or task. A test made up of algebra questions given to an applicant for a maths teacher's position would demonstrate content validity.
Many employers use employment tests and other selection procedures in making employment decisions. Examples of these tools, many of which can be administered online, include the following:
- Cognitive tests assess reasoning, memory, perceptual speed and accuracy, and skills in arithmetic and reading comprehension, as well as knowledge of a particular function or job;
- Physical ability tests measure the physical ability to perform a particular task or the strength of specific muscle groups, as well as strength and stamina in general;
- Sample job tasks (e.g. performance tests, simulations, work samples, and realistic job previews) assess performance and aptitude on particular tasks. NB CAD skills assessments fall into this category;
- Medical inquiries and physical examinations, including psychological tests, assess physical or mental health;
- Personality tests and integrity tests assess the degree to which a person has certain traits or dispositions (e.g. dependability, cooperativeness, safety) or aim to predict the likelihood that a person will engage in certain conduct (e.g. theft, absenteeism);
- Criminal background checks provide information on arrest and conviction history;
- Credit checks provide information on credit and financial history;
- Performance appraisals reflect a supervisor’s assessment of an individual’s performance; and
- English proficiency tests determine English fluency.
An important item to remember when interpreting CAD assessment scores is to put the results in context, and to compare them to an external performance benchmark. For example, a 58% score does not reflect ‘failure’. If this score is presented by a CAD user against an office average of 60%, it is likely the candidate will comfortably fit in to the team.
Test questions should also have a recognizable skill level; basic, intermediate or advanced. In this way, the benchmark data can be reliably used to compare CAD performance and make sound hiring decisions.
In conclusion, for a CAD skills test to be valid, it must contain content that reflects a representative sample of the target skill. A reliable CAD skills test should comprise sample job tasks (e.g. performance tests, simulations, work samples, and realistic job previews) and assess performance and aptitude on particular tasks.
Skills evaluations should not discriminate according to; race, color, national origin, sex, religion, age (40 or older), or disability.
Lastly, employers should ensure that employment tests and other selection procedures are properly validated for the positions and purposes for which they are used. The test or selection procedure must be job-related and its results appropriate for the employer's purpose.
More information can be found here:
http://www.uniformguidelines.com/uniformguidelines.html and
http://www.eeoc.gov/policy/docs/factemployment_procedures.html
Rory
The short answer is yes, in our opinion, but only under the right circumstances. Let’s examine some of the evidence.
The most important issue underlying the use of pre-employment assessments is validity. The question we need to ask is this; ‘Is the test valid for this intended purpose; does it support the decisions that are going to be made?’
So what is 'validity'? Validity measures how appropriate a test is for a specific purpose. Simply put, a test may be considered valid for one use and invalid for another. Why do pre-employment tests need to be validated? In 1978, the Equal Employment Opportunities Commission (EEOC) created guidelines to ensure that the knowledge gained from testing is applied with impartiality to protect minority applicants from discriminatory employment procedures. What is the best method of validation? The EEOC guidelines do not state that one method is better than another; the method used must fit the needs of the business or organization.
There are three methods of validation set forth by the EEOC:
1) Criterion Validity: If data demonstrates that a test is significantly correlated with a vital measure of job performance, the test is said to demonstrate criterion validity. For example, if all the CAD users that scored highly on a selected test to measure CAD skills completed their projects accurately and on time, the test would demonstrate criterion validity.
2) Construct Validity: The term construct is a technical term for personality traits like intelligence and creativity. Construct validity is demonstrated if a test measures traits that have been found to influence successful performance of a job. A test that measures the interpersonal communication skills of a potential customer service representative would demonstrate construct validity.
3) Content Validity: This is demonstrated if the questions that make up an assessment are representative of content that is required to perform a particular activity or task. A test made up of algebra questions given to an applicant for a maths teacher's position would demonstrate content validity.
Many employers use employment tests and other selection procedures in making employment decisions. Examples of these tools, many of which can be administered online, include the following:
- Cognitive tests assess reasoning, memory, perceptual speed and accuracy, and skills in arithmetic and reading comprehension, as well as knowledge of a particular function or job;
- Physical ability tests measure the physical ability to perform a particular task or the strength of specific muscle groups, as well as strength and stamina in general;
- Sample job tasks (e.g. performance tests, simulations, work samples, and realistic job previews) assess performance and aptitude on particular tasks. NB CAD skills assessments fall into this category;
- Medical inquiries and physical examinations, including psychological tests, assess physical or mental health;
- Personality tests and integrity tests assess the degree to which a person has certain traits or dispositions (e.g. dependability, cooperativeness, safety) or aim to predict the likelihood that a person will engage in certain conduct (e.g. theft, absenteeism);
- Criminal background checks provide information on arrest and conviction history;
- Credit checks provide information on credit and financial history;
- Performance appraisals reflect a supervisor’s assessment of an individual’s performance; and
- English proficiency tests determine English fluency.
An important item to remember when interpreting CAD assessment scores is to put the results in context, and to compare them to an external performance benchmark. For example, a 58% score does not reflect ‘failure’. If this score is presented by a CAD user against an office average of 60%, it is likely the candidate will comfortably fit in to the team.
Test questions should also have a recognizable skill level; basic, intermediate or advanced. In this way, the benchmark data can be reliably used to compare CAD performance and make sound hiring decisions.
In conclusion, for a CAD skills test to be valid, it must contain content that reflects a representative sample of the target skill. A reliable CAD skills test should comprise sample job tasks (e.g. performance tests, simulations, work samples, and realistic job previews) and assess performance and aptitude on particular tasks.
Skills evaluations should not discriminate according to; race, color, national origin, sex, religion, age (40 or older), or disability.
Lastly, employers should ensure that employment tests and other selection procedures are properly validated for the positions and purposes for which they are used. The test or selection procedure must be job-related and its results appropriate for the employer's purpose.
More information can be found here:
http://www.uniformguidelines.com/uniformguidelines.html and
http://www.eeoc.gov/policy/docs/factemployment_procedures.html
Rory
Thursday, 27 November 2008
People don't know...what they don't know!
The basic psychology of how people learn new skills is pretty fascinating - really!
Being in the business of measuring CAD & BIM skills of Autodesk & Bentley users, we find ourselves at the heart of the learning debate.
Part of the problem with traditional CAD training is that it can be a bit one-dimensional. Often, over the course of 1, 2, or 3 days, a variety of users from differing backgrounds, and with varying degrees of experience and ability, will gather in a classroom and attempt to absorb every detail of the latest release of AutoCAD, Revit, or whatever, in a torrent of technical information. Inevitably, the trainer has to gravitate to the speed of the slowest learner, and everyone else follows along at that pace. The assumption is (incorrectly!) made that all delegates possess a minimum level of basic CAD ability. Further, the material tends to be off-the-shelf or generic, with individual workflows and processes seldom covered; this can make it difficult for students to relate to the material and apply the information back at the workplace.
Now, consider the following evidence from Rebecca Rupp, author of 'Committed to Memory';
'One hour after learning, 56% of material has gone to the wind; one day later, 66% has evaporated; and after one month, 80% is gone'.
With this in mind, is it any wonder why firms struggle to get a satisfactory bang for their training buck?! Should even a modest amount of time pass between the course ending and real work commencing, it is small wonder that much of the new material isn't applied on the job; the 'use it or lose it' factor.
Over the past four years, CADsmart has compiled results data from over 10,000 live CAD skills assessments. Analysis of the data makes for enlightening reading. By dividing the results into four performance 'quartiles', an interesting trend emerges. Based on accuracy & speed, and against a 'mean' score of 63% in 74 minutes, the following breakdowns occur; 37% comprise the upper quartile (4), which means they are faster and more accurate than the average. 1 in 5 users make it into the mid-upper quartile (3); more accurate but slower than the mean. Another 20% populate the mid-lower quartile (2); less accurate but faster than their peers. Finally, 23% of users (nearly a quarter!) make up the lower quartile (1), performing slower and with less quality than the rest.
In our experience, until they complete a formal skills assessment, most users have no real idea of how skilled they are, relative to their peers in industry. Many claim to be 'experts' or 'super users', based on nothing more than their years of service with Autodesk or Bentley tools! Simply put, they don't know what they don't know.
So what journey do we take, as we progress our skills from basic, through intermediate and advanced levels? The 'Conscious Competence' model explains the process and stages of learning a new skill (behaviour, ability or technique).
Stage one is 'Unconscious Incompetence'; a person is not aware of the existence or relevance of the skill and is unaware that they have a particular deficiency in the area concerned.
Next, we progress to 'Conscious Incompetence'; a person becomes aware of the existence and relevance of the skill and is therefore also aware of their deficiency in this area, ideally by attempting or trying to use the skill, or by formal assessment. The person is helped by evaluating the extent of their deficiency in the relevant skill, and a measure of what level of skill is required for their own competence.
A person achieves 'Conscious Competence' in a skill when they can perform it reliably at will; they need to concentrate in order to perform the skill - it is not yet 'second nature' or automatic and it is unlikely they will be able to teach it well to others.
Finally, with practise and persistence, the skill becomes so familiar that it enters the unconscious parts of the brain - 'Unconscious Competence' is achieved. Typical examples are driving, sports activities, typing, manual dexterity tasks - such as CAD! - listening and communicating. The skill becomes 'second nature', but arguably gives rise to the need for long-standing competence to be checked periodically against new standards.
We have a saying at CADsmart; 'If you can't measure it, you can't manage it'. By putting in place reliable performance metrics, including a formal assessment program, AEC firms can make the transition from basic 2D drafting to more complex 3D modelling and BIM skills with a higher degree of confidence.
Rory
Being in the business of measuring CAD & BIM skills of Autodesk & Bentley users, we find ourselves at the heart of the learning debate.
Part of the problem with traditional CAD training is that it can be a bit one-dimensional. Often, over the course of 1, 2, or 3 days, a variety of users from differing backgrounds, and with varying degrees of experience and ability, will gather in a classroom and attempt to absorb every detail of the latest release of AutoCAD, Revit, or whatever, in a torrent of technical information. Inevitably, the trainer has to gravitate to the speed of the slowest learner, and everyone else follows along at that pace. The assumption is (incorrectly!) made that all delegates possess a minimum level of basic CAD ability. Further, the material tends to be off-the-shelf or generic, with individual workflows and processes seldom covered; this can make it difficult for students to relate to the material and apply the information back at the workplace.
Now, consider the following evidence from Rebecca Rupp, author of 'Committed to Memory';
'One hour after learning, 56% of material has gone to the wind; one day later, 66% has evaporated; and after one month, 80% is gone'.
With this in mind, is it any wonder why firms struggle to get a satisfactory bang for their training buck?! Should even a modest amount of time pass between the course ending and real work commencing, it is small wonder that much of the new material isn't applied on the job; the 'use it or lose it' factor.
Over the past four years, CADsmart has compiled results data from over 10,000 live CAD skills assessments. Analysis of the data makes for enlightening reading. By dividing the results into four performance 'quartiles', an interesting trend emerges. Based on accuracy & speed, and against a 'mean' score of 63% in 74 minutes, the following breakdowns occur; 37% comprise the upper quartile (4), which means they are faster and more accurate than the average. 1 in 5 users make it into the mid-upper quartile (3); more accurate but slower than the mean. Another 20% populate the mid-lower quartile (2); less accurate but faster than their peers. Finally, 23% of users (nearly a quarter!) make up the lower quartile (1), performing slower and with less quality than the rest.
In our experience, until they complete a formal skills assessment, most users have no real idea of how skilled they are, relative to their peers in industry. Many claim to be 'experts' or 'super users', based on nothing more than their years of service with Autodesk or Bentley tools! Simply put, they don't know what they don't know.
So what journey do we take, as we progress our skills from basic, through intermediate and advanced levels? The 'Conscious Competence' model explains the process and stages of learning a new skill (behaviour, ability or technique).
Stage one is 'Unconscious Incompetence'; a person is not aware of the existence or relevance of the skill and is unaware that they have a particular deficiency in the area concerned.
Next, we progress to 'Conscious Incompetence'; a person becomes aware of the existence and relevance of the skill and is therefore also aware of their deficiency in this area, ideally by attempting or trying to use the skill, or by formal assessment. The person is helped by evaluating the extent of their deficiency in the relevant skill, and a measure of what level of skill is required for their own competence.
A person achieves 'Conscious Competence' in a skill when they can perform it reliably at will; they need to concentrate in order to perform the skill - it is not yet 'second nature' or automatic and it is unlikely they will be able to teach it well to others.
Finally, with practise and persistence, the skill becomes so familiar that it enters the unconscious parts of the brain - 'Unconscious Competence' is achieved. Typical examples are driving, sports activities, typing, manual dexterity tasks - such as CAD! - listening and communicating. The skill becomes 'second nature', but arguably gives rise to the need for long-standing competence to be checked periodically against new standards.
We have a saying at CADsmart; 'If you can't measure it, you can't manage it'. By putting in place reliable performance metrics, including a formal assessment program, AEC firms can make the transition from basic 2D drafting to more complex 3D modelling and BIM skills with a higher degree of confidence.
Rory
Sunday, 16 November 2008
World first from CADsmart - interactive Revit skills assessment software
Well, it's been a while in the making, but we're just about there. About a year ago, we sat around a table with a half dozen leading UK Revit users from our customer base at HOK's London offices (thanks for the coffee & biccies, Miles!). Our aim was to determine how we should attempt to break down Autodesk's flagship AEC CAD (BIM?) software, so that a meaningful basic skills assessment, complete with accurate training needs reporting, could be developed.
Fast forward to the present, and we're nearing the end of our final beta test phase - and I must confess, we're pretty pleased with the result. Having battled with a somewhat (still) limited API and developer environment, we are on the cusp of launching a world first - a fully interactive basic Revit skills assessment. I should say that Autodesk's support team has been very helpful in responding to a host of technical support issues from our team - and things can only get better as they continue to tick things off their own Revit development wish list.
Our timing is just about spot on, as our goal was to launch the new software at Autodesk University 2008, which is now just around the corner. In fact, Tony & I will be teaching a class at this year's event, appropriately titled, 'How to make the Move from AutoCAD to Revit'. By all means come along! :)
In anticipation of the launch of the software, we are also sponsoring the show, so here's the plug; we'll be at booth 237 - and the first 100 firms to sign up for a demo of the Revit skills software will - if they like it - qualify for a 50% discount on their first year's subscription!
In anticipation of the launch of the software, we are also sponsoring the show, so here's the plug; we'll be at booth 237 - and the first 100 firms to sign up for a demo of the Revit skills software will - if they like it - qualify for a 50% discount on their first year's subscription!
Look forward to meeting you in Vegas!
Rory
Tuesday, 4 November 2008
Language and progress
With the imminent release of our skills assessment software for Autodesk Revit, we have been musing here at CADsmart Towers as to whether the term 'BIM' or some other acronym might ever fully usurp the term 'CAD' and make our name and corporate identity look a little dated.
But today it struck me - it doesn't matter, because one of Europe's biggest companies has a corporate name that is so dated due to a rapid change in technology that it would be laughable if it were proposed for a new firm today, yet they continue to dominate in their field with a very clear message about what they do.
Step forward The Carphone Warehouse. If you go to their website and search for 'car phone' you get no results - they don't sell any. Of course they don't, and everyone knows this. Consumers are so familiar with their high-street name that nobody thinks twice about it.
Long live CADsmart!
Ed
Subscribe to:
Posts (Atom)
Posts
