Tool Evaluation Criteria
Below is a list of robust criteria presented as key questions to evaluate the pros and cons of an edtech tool in this context. While no tool excels in every area, the importance of each criterion depends on the type of tool, implementation goals, and context. You can make your own copy of the associated Workforce EdTech Tool Evaluation Scoring Rubric and Notepad (an editable Google Sheet) to take notes, score, and make decisions with your team about adopting a certain edtech tool for the workforce development and adult foundational literacy context.
Criteria for Evaluating Workforce EdTech Tools:
To Increase Reach and Impact of Adult Learning & Employment Initiatives
Proven Effectiveness
Is there evidence that this technology can help meet your goals?
- What evidence exists to prove the effectiveness of the tool?
- To what extent does this evidence suggest that the tool could help meet your specific goals?
- What other organizations have effectively leveraged the tool? What factors are different or similar to yours (setting, users, goals, etc.)?
- What outcomes can you expect, and over what period of time?
Accessibility
Is the Technology accessible and easy to use for all learners?
Ease of Use
- How much training will the user need to use the tool effectively? What are the steps between purchase and active use, and how does the vendor support that process?
- Is the tool built using best practices in Universal Design so that it can be accessed, understood, and used by all people to the greatest extent possible?
- Are instructions easy to find and follow? Is the layout clear for easy navigation?
- Can users freely navigate, backtrack, and explore the content if helpful or needed?
Digital Inclusion
- How does the tool accommodate varying levels of internet connectivity, including full online, intermittent, and offline usage scenarios?
- How does the tool support users with different digital literacy levels, including registration requirements, initial training, ongoing support, navigational cues, and integration with familiar tools or interfaces?
- Can the tool be easily accessed on multiple devices including desktop and mobile, and on various operating systems (Android, Windows, iOS, etc.)?
Language and Culture
- How well does the tool address language needs, including translations into users’ first languages, consistency and accuracy of language access, and appropriateness of text complexity for the intended users’ literacy levels?
- Does the tool’s content reflect cultural diversity, avoid bias, and appropriately represent the cultural contexts of its users?
Affordability
What is the cost and is it affordable now, during scaling, and in the long-term?
- What is the pricing model for the tool? Are there subscription models or pricing tiers, and how might they impact the tool’s affordability in the long run?
- What are the primary cost considerations for the tool, including long-term updates, maintenance, and any required additional investments (staff training, e.g.)?
- Will the tool remain affordable as you scale its use over the long term?
User Experience
Is the tool enjoyable and used effectively by the intended audience to meet their goals?
Onboarding, Hook & Engagement
- How seamless and engaging is the initial login and introduction to the tool?
- What strategies are implemented to maintain user engagement through diverse content forms and interactive features?
- Are the contents up-to-date, accurate, relevant, and effectively aligned with users’ goals?
- How is content organized and presented to ensure it is digestible, logically structured, and supports seamless transitions between activities?
User-Centered Design and Feedback
- Does the tool prioritize and effectively meet the needs of its primary users, adapting to their interactions and avoiding biases?
- What do user testimonials or external reviews indicate about its success in meeting these needs?
- How do the vendors engage end-users in the design and evaluation process?
User Support & Communication
Does the tool provide adequate and accessible user support and communication?
- What is the onboarding process and what are the resources the tool provides for training new users to use it?
- Does the tool provide support for users? Is the tech support easily accessible and sufficient?
- Are there accessible and effective communication methods for updating users and keeping them engaged?
Data, Privacy & Security
Is the solution set up to give you the data you need in a sustainable way and are you clear and comfortable with the privacy policies?
Data Access
- Does the tool provide the data you need for your intended purposes related to its intended use and measurement of impact?
- Do the tool’s reporting functions meet your needs and the needs of the end-users?
- Can the data be easily accessed, visualized, and aligned to measurable goals?
- Is the data collected and exported in a way that enables data interoperability with other tools and systems?
Privacy & Security
- What data is shared, how is it shared, and with whom?
Are users effectively and appropriately made aware of data policies and practices in a way that is easy to understand and explain? - Does the tool or connected tools collect, track, or otherwise leverage Personally identifiable information (PII)?
Longevity
Are you confident that the vendor will continue to exist and provide needed support and updates to the product for the period of time you may need it?
- How long has the tech solution/ developer been in business and at what development stage is the product (BETA or fully developed)?
- How committed is the provider to regularly updating the tool and incorporating new advancements?
- What strategies does the provider have in place for continuous research and technological advancement?
Criteria for Evaluating Workforce EdTech Tools by Ed Tech Center @World Education, Innovate & Educate, Generation.org, and numerous other organizations and individuals is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Do you have a comment or a question?