Module 12 Content: Concerns of AI
In previous sections, we introduced what AI systems are and how they work, along with basic ways of interacting with them in practice. This section shifts attention to their limitations and risks. As AI becomes more common in everyday contexts, it is necessary to examine not only what these systems can do, but also the constraints and tradeoffs they bring.
Some concerns are visible in the output itself, such as hallucinated facts or misleading summaries. Other concerns are less visible, such as privacy, copyright, data collection, or environmental cost.
AI literacy therefore requires critical evaluation. Users need to consider what data a system relies on, what risks may arise from its use, and what must be verified before trusting or reusing its output.
The concerns introduced here are not a complete list. They are examples of major issues that appear across many current AI systems.
Environmental cost
AI systems require physical infrastructure. They depend on data centers, computing hardware, electricity, cooling, and networked services. For this reason, AI is not only a digital software. It also has material and environmental costs.
These costs are not limited to building a model. Running AI systems at scale also consumes resources. A tool may feel quick and convenient to an individual user while still relying on large amounts of energy in the background.
At the same time, views on this issue are not one-sided. Some people are concerned that AI systems consume too much energy and place additional strain on existing infrastructure. Others argue that AI may help address energy challenges by improving efficiency, optimizing resource use, or supporting research in areas such as energy systems and climate modeling.
Further information
- Business Insider: Exposing The Dark Side of America’s AI Data Center Explosion
- Sims Witherspoon: Can AI Help Solve the Climate Crisis?
- Pew Research: What we know about energy use at U.S. data centers amid the AI boom
Privacy and data security
Many AI systems collect and process user input, including prompts, uploaded files, and metadata. Users may not know what is stored, how long it is retained, or whether it is reviewed or used for model improvement.
Privacy therefore becomes a core concern in AI literacy. Ease of use does not imply appropriateness. Sensitive personal, academic, workplace, legal, or health-related data require caution.
Privacy is closely tied to security. A tool’s interface may appear trustworthy while its underlying processes remain unclear. In professional contexts, the key question is not only whether a task can be done, but whether the data should be entered into the system at all.
For information organizations, this issue is critical. Libraries, universities, and archives often handle confidential or protected records. A convenient AI tool is not always an appropriate one.
Think about how you usually use tools like ChatGPT, Gemini, Claude, or Copilot.
Consider:
What kinds of information have you entered into these tools?
When you type something, what do you think happens to that data?
Do you assume:
- it is stored?
- it is used to improve the system?
- it disappears after the session?
Now think more concretely:
Have you ever entered anything that could be:
- personal
- student-related
- workplace or institutional
- sensitive or confidential
Reflection:
- Before this, did you actively think about where your data goes?
- What assumptions were you making?
- Do you feel comfortable with those assumptions now?
Goal:
Understand how AI tools you actually use handle your data, and what control you have over that process.
Choose one AI tool that you personally use or are likely to use. Go to its privacy, data, or settings page and explore how it handles user data (e.g., ChatGPT, Gemini, Claude, or others).
As you explore, focus on the following questions:
- Can you turn off the use of your data for model training?
- If you turn it off, does anything else change (e.g., chat history, features, integrations)?
- Can you delete past conversations or activity?
- Is any data still temporarily stored even after settings are turned off?
After exploring your chosen tool, briefly reflect on:
- How much control does this tool actually give you over your data?
- Are the settings easy to find and understand?
- Are there any tradeoffs (e.g., losing features vs. protecting privacy)?
Note: These settings and policies can change over time. Always rely on the current official documentation rather than screenshots or summaries.
Further information
Hallucinations and confabulations
Generative AI can produce fluent and convincing text even when it is wrong.
Some errors are easy to spot, such as invented sources or incorrect facts. Others are subtle, such as summaries that distort meaning or citations with small but critical mistakes. Outputs may also mix correct and incorrect information.
Fluency can mislead. Clear and confident language may be trusted too quickly, especially in academic or professional contexts.
Reliability is therefore a literacy issue. Users need to verify claims, check sources, compare outputs, and recognize uncertainty.
Further information
Copyright, ownership, and training data
AI systems raise questions about copyright, licensing, and ownership, both for training data and for generated output.
Training data is often drawn from large collections of text, images, audio, or code. Users usually do not know what was included, whether permission was obtained, or how materials were licensed.
In academic publishing, this issue is already reflected in policy. Many publishers, such as Springer, do not allow AI systems to be listed as authors. AI tools may be used in the research or writing process, but responsibility for the content must remain with human authors. Also see COPE’s position.
Further information
Bias, fairness, transparency, and explainability
AI systems can reflect bias in the data they are trained on and in the ways they are designed or evaluated. As a result, a system may appear neutral while still producing uneven outcomes.
This matters for fairness, transparency, and explainability. If users are affected by a system without understanding how it works, trust and accountability become harder. In information organizations, this is especially relevant in areas such as search, ranking, recommendation, and decision support.
Further information
AI and labor
AI can affect how work is done. It is often used to increase speed or reduce routine effort, but it can also reshape jobs in uneven ways.
One concern is deskilling. If workers rely heavily on AI for tasks such as writing, coding, searching, or analysis, they may practice these skills less often. Over time, this can reduce their ability to perform tasks independently or to evaluate AI output.
AI can also influence hiring, expectations, and workload. In some settings, workers may be expected to do more in less time because AI tools are available.
Further information
Other concerns
The concerns discussed above are not the only ones. AI systems can also raise other issues that deserve attention, especially when they are adopted broadly across education, workplaces, and public communication.
One concern is overreliance. Students or workers may begin using AI not only as a support tool, but as a default way of completing tasks, such as reading and writing. Excessive reliance on AI may lead to psychosis.
Another concern is unequal language support. Many AI systems perform much better in high-resource languages such as English than in low-resource languages. For example, translation quality may be noticeably weaker, or search results may be less reliable. This can limit access and reduce the usefulness of AI tools for some users. As a result, AI may expand access in some settings while reinforcing inequality in others.
Further information