As artificial intelligence (AI), particularly generative AI, becomes increasingly woven into our professional and personal lives—from personalized travel itineraries to reviewing resumes to summarizing investigation notes and reports—questions about who or what controls our data and how it’s used are ever present. AI systems survive and thrive on information and that intersection of AI and privacy elevates the need for data protection.
Recent regulations issued by the California Privacy Protection Agency (CPPA) under the California Consumer Privacy Act (CCPA) begin to erect those protections. Among its various provisions, the CCPA now specifically addresses automated decision-making technologies (ADMT), attempting to bring transparency and consumer rights to, among other things, push back on algorithms making significant decisions about them.
As a starting point, it is important to define ADMT. Under the CCPA, it means any technology that processes personal information and uses computation to replace human decision-making or substantially replace human decision-making. For this purpose, “replace” means to make decision without human involvement. To be considered human involvement, a human must:
- know how to interpret and use the technology’s output to make the decision;
- review and analyze the output of the technology, and any other information that is relevant to make or change the decision; and
- have the authority to make or change the decision based on their analysis in (B).
CCPA-covered businesses that use ADMT to make “significant decisions” about consumers have several new compliance obligations to navigate. A “significant decision” is defined as a decision that has important consequences for a consumer’s life, opportunities, or access to essential services. CCPA regulations define these decisions as those that result in the provision or denial of:
- Financial or lending services (e.g., credit approval, loan eligibility)
- Housing (e.g., rental applications, mortgage decisions)
- Education enrollment or opportunities (e.g., admissions decisions)
- Employment or independent contracting opportunities or compensation (e.g., hiring, promotions, work assignments)
- Healthcare services (e.g., treatment eligibility, insurance coverage)
These decisions are considered “significant” because they directly affect a consumer’s economic, health, or personal well-being.
When such businesses use ADMT to make significant decisions, they generally must do the following:
- Provide an opt-out right for consumers.
- Provide a pre-use notice that clearly explains the business’s use of ADMT, in plain language.
- Provide consumers with the ability to request information about the business’s use of ADMT.
Businesses using ADMT for significant decisions before January 1, 2027, must comply by January 1, 2027. Businesses that begin using ADMT after January 1, 2027, must comply immediately when the use begins.
Businesses will need to examine these new requirements carefully, including how they fit into the existing CCPA compliance framework, along with exceptions that may apply. For example, in the case of a consumer’s right to opt-out of ADMT, a business may not be required to make that right available.
If a business provides consumers with a method to appeal the ADMT decision to a human reviewer who has the authority to overturn the decision, opt-out is not required. Additionally, the right to opt-out of ADMT in connection with certain admission, acceptance, or hiring decisions, is not required if the following are satisfied:
- the business uses ADMT solely for the business’s assessment of the consumer’s ability to perform at work or in an educational program to determine whether to admit, accept, or hire them; and
- the ADMT works as intended for the business’s proposed use and does not unlawfully discriminate based upon protected characteristics.
Likewise, the right to opt-out of ADMT is not required for certain allocation/assignment of work and compensation decisions, if the business:
- uses the ADMT solely for the business’s allocation/assignment of work or compensation; and
- the ADMT works for the business’s purpose and does not unlawfully discriminate based upon protected characteristics.
As many businesses are realizing, successfully deploying AI requires a coordinated approach to achieve more than getting the desired output. It includes understanding a complex regulatory environment of which data privacy and security is a significant part.
