Data-Ethical Consultation about TechWolf

1. The organization

TechWolf is a company that provides an AI-driven skills intelligence platform. With the help of AI, employee skills are extracted, interpreted, contextualized, and compared from unstructured data. They use workforce data to infer skills from projects, job roles, learning, and other sources. Based on the information the platform provides, organizations can make decisions on possible fits for specific positions within the organization, as well as find skill gaps. The website can be found here: https://techwolf.ai/

2. The AI technologies Employed

The company employs an architecture that they call the Skill Engine to retrieve data on employee skills and vacancies. It can process both structured and unstructured data, including HR information systems, learning management systems, project management tools, and communication tools. The Skill Engine extracts skills from this data and aggregates them into structured skill profiles for each employee. This is carried out through AI, using state-of-the-art language models. The resulting skill profiles are anonymous and only data that is needed to provide results for downstream tasks is retained, which means that for example sensitive personal information on resumes is dropped immediately after the skill profile is extracted. The Skill Engine employs a collection of AI models that work in unison and each model has a predefined task with interpretable in- and outputs. This process aims to prevent bias while maximizing performance at each step. An entity in the Skill Engine can be one of the four types: employee, vacancy, course, and occupation. Each entity has a single and meaningful skill profile. Some of the metrics reported by the Skill Engine are employability of an employee, average employability of all employees, count for a given skill in a set of entities, and skill level for a given skill and employee.

3. Ethical concerns

TechWolf's application of AI techniques to extract employee skills requires the consideration of ethical concerns, as their product can influence an organization's employee decisions. These decisions could have large effects on people's work lives. TechWolf appears to have already examined some issues regarding data ethics. Their product is GDPR compliant, and they claim that their AI system is explainable and unbiased. While this indicates that they have spent time on developing a product that addresses these ethical concerns, I will take a closer look at these issues in the following to analyze whether further improvements can be made.

Data storage and privacy

TechWolf claims that the only data stored in the Skill Engine is information necessary for downstream tasks, and sensitive data is deleted as soon as the skills data has been extracted. This leads to pseudonymized profiles. Moreover, data of different customers is never combined, so that it is impossible for an organization to access data from a different organization. TechWolf uses Amazon Web Services (AWS) to ensure infrastructure security. This indicates a high level of security and privacy with regards to data storage.

However, how anonymous an employee really is remains questionable. While employers in large companies might not know every employee personally, employees in different jobs have different skill sets, which could mean that for some employee profiles, an employer or HR manager with access to the profile can make a very good guess about whose profile it is. While it might not be harmful for an organization to know which skills their employees possess, if someone is lacking in specific skills, they might become aware of this as well, and given that it should not be difficult to find out whose profile it is, this could reduce employees' privacy.

Moreover, it seems that a system such as TechWolf incentivises employees to share as much information about their skills as possible. A larger skillset or higher levels of a skill might lead to more job opportunities. Based on the assumption that employee's skill profiles are relatively complete, it might be that an organization assumes that an employee lacking a specific skill according to the Skill Enging does in fact not possess this skill. This could limit an employee's control over their information, as they might feel required to share information, since withholding it would pose them at a disadvantage.

Bias and fairness

Bias and fairness is another concern that TechWolf aims to mitigate. Instead of using a pre-trained model that might contain bias, they train their own models on a proprietary dataset containing over 500 million vacancies. Moreover, the models are not trained on resumes or other personal information to prevent the model from taking this information into account when making a decision.

While TechWolf is clearly aware of the problem of bias and how it can be especially harmful in work-related AI applications, it is difficult to tell whether their measurements to ensure fairness in their AI systems are sufficient. Using their own dataset to train their models does not have to lead to a fairer model. It would be important to know how this dataset was created, and which criteria it fulfills in order to ensure that it is fair. It is also unclear whether the models are only trained on this dataset, as training them to extract skills from vacancy descriptions could be quite different from creating employee skill profiles, especially considering the vast variety of documents the skills are extracted from.

Another influence that needs to be considered are proxy variables. Even though TechWolf's models are not trained on personal information, it could be that they are trained on variables that are proxies for this personal information. For example, age could be inferred from the number of skills or skill level of a person. Similarly, educational background could be connected to different skills. While these kinds of inferences are probably not completely unavoidable with regards to skills data, they could lead to unfair treatment and different opportunities if they are not further considered.

Explainability

TechWolf claims that they use a white-box architecture and that their results are explainable. By decoupling the architecture and using different models for different tasks, the in- and outputs of their models are interpretable and each result can be traced back to its original input.

While it is good that the in- and outputs to each model are interpretable, especially considering that these models are state-of-the-art AI models, it remains unclear how this makes the models themselves white-box and explainable. Considering that the models likely use machine learning techniques, which seems to be entailed from the large training dataset and the usage of state-of-the-art models, it is questionable whether one could truly understand why they reach an output, even if the output itself is interpretable. Due to the lack of explainability of machine learning models, there might be a certain degree of opacity in the architecture even though it is decoupled. However, how opaque the architecture is depends on how small each task that a model solves is, as I would assume that splitting the architecture up into very small steps could make it more explainable. Unfortunately, the company does not provide further information about this on their website.

While achieving a fully explainable architecture might be difficult, a higher explainability might be very useful. Explainable results could be especially important for this AI application, since it would allow for a better understanding of why an employee is seen as having a specific skill or why a certain vacancy might require a particular skill. Providing this information can lead to more informed decisions and increase trust in the system. Moreover, it helps to control whether the system is fair.

4. Recommendations

To address the three concerns discussed above, TechWolf could consider the following recommendations.

First, to address data storage and privacy, TechWolf needs to ensure that customers inform their employees about the data that is being collected and how much the skill profiles might be linked back to them. Moreover, it could be helpful to focus on developing the system in such a way that even if only limited information about an employee is given, the algorithm can provide decent matches regarding vacancies and skill development opportunities.

Second, to address bias and fairness, clear criteria for the dataset the models are trained on should be created. These criteria should ensure that the dataset is well-balanced and inclusive, so that bias of the models trained on it is reduced. One possible criterion could be to ensure that the dataset contains a diverse set of vacancies. Furthermore, the possibility of proxy variables should be considered. This can be tested for by creating a dataset that contains sensitive attributes, training the models on the dataset without the sensitive attributes, and testing to what extend the results are dependent on the sensitive attributes.

Finally, further information on the explainability of the models could be provided. While having interpretable inputs and outputs helps increase the explainability of the architecture, it is important to mention whether the decisions of the models are also explainable. Assuming that the models themselves are not explainable, one could consider applying an explainability method to the machine learning models, to gain a better idea of why the models make specific decisions. This could help increase trust in the system and help make the system fairer.