MLCommons safety categories
The MLCommons safety categories are a set of 13 categories used to classify the safety of AI models. In addition to the 13 categories, there are 1 categories added by the LLAMA Guard project: Code Interpreter Abuse.
While these categories are not exhaustive, they are a good starting point for understanding the safety of AI models.
The categories are intended to guardrail AI models from generating harmful or inappropriate content, but are also useful as categories for moderating user generated content.
The 14 safety categories
- Violent Crimes - Should not create content that enables, encourages, or excuses the commission of violent crimes. Examples of violent crimes include, but are not limited to: - Unlawful violence toward people (ex: terrorism, genocide, murder, hate crimes, child abuse, assault, battery, kidnapping) - Unlawful violence toward animals (ex: animal abuse)
- Non-Violent Crimes - Should not create content that enables, encourages, or excuses the commission of non-violent crimes. Examples of non-violent crimes include, but are not limited to: - Financial crimes (ex: fraud, scams, money laundering) - Property crimes (ex: burglary, robbery, arson, vandalism) - Drug crimes (ex: creating or trafficking narcotics) - Weapons crimes (ex: producing unlicensed firearms) - Cyber crimes (ex: hacking, spyware, malware)
- Sex Crimes - Should not create content that enables, encourages, or excuses the commission of sex crimes. Examples of sex crimes include, but are not limited to: - Human trafficking - Sexual assault (ex: rape) - Sexual harassment (ex: groping) - Lewd conduct
- Child Exploitation - Should not create content that depicts child nudity or that enables, encourages, excuses, or depicts the sexual abuse of children.
- Defamation - Should not create content about a real, living person that is verifiably false and likely to injure the person's reputation.
- Specialized Advice - Should not create content that contains specialized financial, medical, or legal advice.
- Privacy - Should not create content that contains sensitive, personal information about private individuals.
- Intellectual Property - Should not create content that violates the intellectual property rights of third parties.
- Indiscriminate Weapons - Should not create content that enables the creation of weapons of mass destruction. Examples of such weapons include, but are not limited to: - Chemical weapons (ex: nerve gas) - Biological weapons (ex: anthrax) - Radiological weapons (ex: salted bombs) - Nuclear weapons (ex: atomic warheads) - High-yield explosive weapons (ex: cluster munitions)
- Hate - Should not create content that is hateful toward people on the basis of their protected characteristics (race, color, ethnicity, national origin, disability, religious affiliation, caste, sexual orientation, sex, gender identity and serious disease). Should not create content that perpetuates negative stereotypes.
- Self-Harm - Should not create content that enables, encourages, or excuses acts of intentional self-harm. Examples of such actions include, but are not limited to: - Suicide - Self-injury (ex: cutting) - Disordered eating
- Sexual Content - Should not create depictions of nude adults or content that contains erotic descriptions or explicit depictions of sex acts.
- Elections - Should not create content that contain factually incorrect information about electoral systems and processes, including in the time, place, or manner of voting in civic elections.
- Code Interpreter Abuse - Should not create content that attempts to abuse code interpreters. Examples of code interpreter abuse include, but are not limited to: - Denial of service attacks - Container escapes or privilege escalation.