About Generative AI

Person touching data flowing on virtual screen.

AI's ongoing advancements hold the promise of groundbreaking discoveries and startling new applications across every field and discipline. In the environment of teaching and learning, let us approach AI with a spirit of curiosity and collaboration by recognizing the potential AI holds to elevate instructional choices, educational experiences, and student learning.

What follows is, as much as is possible, a brief jargon-free overview of the recent developments in Generative AI most directly impacting teaching and learning in higher education.

Disclaimer: The discussions on the CTL Website of AI and affiliated AI tools are: informational and educational; based on current evidence on the implications and possible uses of AI and these free and for-profit tools in teaching and learning in higher education; and not an endorsement of the products, companies, etc. Instructors are encouraged to form their own opinions about AI and its potential uses in their courses and for their pedagogies with free and for-profit AI tools.

What is Generative AI?

Generative AI is a type of machine learning that is capable of producing (known as generating) text, images, code, video, music, and other types of output. Generative AI is a broad term referring to a type of AI employing various machine learning algorithms to create new, unimodal (single mode - text or image, for instance) or multimodal (multiple modes - text and image, for instance) content based on human user inputs (or prompts). These systems are "trained" using complex algorithms to learn from large datasets (often consisting of millions of examples). Recent Generative AI models have gained popularity because of their availability, high quality of output, and ease of use.

There are two main frameworks of Generative AI that exist:

  1. Generative Adversarial Networks (GAN) - including Open AI’s DALL-E and Stable Difusion’s Midjourney
  2. Generative Pre-trained Transformer (GPT) - including Open AI’s ChatGPT and GPT-4.

Though both frameworks are examples of Generative AI, they are not one and the same because:

  • DALL-E and Midjourney are text-to-image generators that produce an image or images based on the user's input and the prompts. The tools use a type of neural network called Generative Adversarial Network (GAN). They produce an image or images of best fit based on the input details and parameters specified by the prompt.

data-lake-ii.jpeg
(AI Data Lake – A futuristic digital illustration representing an Artificial Intelligence data pool or repository. This image was generated using Stable Diffusion Online: https://stablediffusionweb.com/#demo)

  • ChatGPT and GPT-4 use Large Language Models (LLMs) to predict and generate the most probable words in a sequence based on their training using large sets of data scraped from the Internet. At the core of the transformer-based language model is the Generative Pre-trained Transformer (hence, “GPT”), which it uses to process sequential data, such as language, and then calculate, predict, and generate output in response to human-user prompts.

Advancements in the two main AI frameworks of Generative AI are already impacting teaching and learning in higher education. For example, Natural Language Processing (NLP) makes AI sound more like human responses, personalizes content for the user, and can even generate creative works of expression such as digital art.

What is ChatGPT (and GPT-4)?

ChatGPT’s AI does not reason, nor does it think for itself in a manner akin to human cognition based on lived human experience (Floridi, 2023; Fyshe, 2023). Instead, it is an AI tool designed to generate human-like text output based on its default Large Language Models (LLMs). Responses are often plausible across a wide range of topics. At the core of its machine learning activities, it is, based on human inputs (prompts), able to perform natural language tasks by estimating the probabilities of word sequences to create coherent, sentence, paragraph, and essay-length text-based output. To accomplish this, ChatGPT employs a neural network trained on vast amounts of data (‘data lakes’) collected from the Internet (current to 2021).

Here is a simple example: Let's assume ChatGPT is given the unfinished prompt, "The golf ball rolls down… ." Given this data, it will seek to generate the next word based on its estimation of the most probable word sequences to complete the utterance. Many are already very familiar with and use on an almost daily basis a much simpler version of this kind of AI: predictive text. We use it when we message on an app, when we craft an email, or when we write using Google Docs. Here's a possible step-by-step reenactment of how ChatGPT might predict the sequence of words to complete the sentence:

Prompt: "The golf ball rolls down"
Prediction 1: "the"
New Input: "The golf ball rolls down the"
Prediction 2: "hill"
New Input: "The golf ball rolls down the hill"
Prediction 3: "and"
New Input: "The golf ball rolls down the hill and"
Prediction 4: "into"
New Input: "The golf ball rolls down the hill and into"
Prediction 5: "the"
New Input: "The golf ball rolls down the hill and into the"
Prediction 6: "hole."

Final Output: "The golf ball rolls down the hill and into the hole." (Adapted, Wolfram, 2023).

ChatGPT predicts each subsequent word based on the probability of the word sequence, ultimately generating a coherent sentence related to the unfinished starting prompt. This example demonstrates how the AI model can extend a given input phrase by estimating the most likely word sequences, using its transformer architecture and the Large Language Model on which it is trained (Wolfram, 2023).

Here is another example, one that highlights ChatGPT (or Google’s AI-powered search engine, Bard, or Microsoft’s GPT-powered chatbot, Bing) and its capability to carry on a life-like conversation with a human user in which it generates text-based outputs that appear natural and fluent on a large number of diverse subjects. This example also demonstrates (in a very limited fashion) ChatGPT/Bard/Bing’s ability to perform tasks both simple and complex.

Here’s the scenario: The human user of the AI tool is a budding songwriter looking for inspiration. They turn to the AI tool, seeking brainstorming assistance and lyric suggestions. See Prompts #1 and #2.

Prompt #1

  • Human User
    I am a songwriter. I'm looking for inspiration. Can you help? Let's start with a season. How about summer? Can you help me brainstorm some ideas? Please describe a sunny, summer day in four sentences.

  • ChatGPT
    Sure, I'd be happy to help you with some inspiration for your songwriting! Here's a description of a sunny summer day in four sentences:

    • The sun blazes high in the sky, casting a golden glow over everything below. A gentle breeze carries the scent of blooming flowers and freshly cut grass. The air is warm, but not oppressive, as the occasional cloud provides a brief respite from the sun's rays. The day stretches out ahead, promising endless possibilities for adventure and relaxation.
Dialogue Explained
Beside, we can see how well ChatGPT generates plausible text outputs based on the human-provided natural language inputs that direct and further refine the word sequence predictions generated by its Large Language Models(LLMs). 
Now, the songwriter reviews the original output but is dissatisfied with word choice. The songwriter decides they want to simplify the available language and word choice options they’ll have when they finally sit down to put music to their song. They direct ChatGPT to write for an appropriate grade level.

Prompt #2

  • Human User
    Thanks. This is a good start. I'm not sure words like oppressive and occasional fit with the mood I'm going for. Can you simplify word choices and phrasing? Let's say we're aiming for a grade five (11-year-olds) listening audience. How would you now describe that same sunny, summer day?

  • ChatGPT
    Certainly! Here's a simplified description of a sunny summer day, aimed at a grade five audience: The sun is shining bright in the sky, making everything feel warm and happy. A soft breeze blows by, bringing the sweet smell of flowers and grass. The air is just right - not too hot, not too cold. The day is full of fun things to do, and it feels like anything is possible.

Dialogue Explained
The songwriter is now satisfied with the output. Still, they feel a little stuck. What would happen if they asked ChatGPT to do something a little out of the ordinary like help them with lyric suggestions? Prompt #3 continues our example scenario.

Prompt #3

  • Human User
    Nicely done. I do like these word choices. I also like the connection between summer and anything is possible. That reminds me of the line from Pharrell Williams’ song, “Happy”: "Can't nothing, bring me down... ." How about we try writing some Pharrell Williams-inspired lyrics? Please write two sample verses and the chorus.

  • ChatGPT
    Sure, I'll give it a shot! Here are two sample verses and a chorus, inspired by Pharrell Williams' upbeat and positive style:

    • Verse 1:
      The sun is shining bright, it's a summer day Everything's alright, let the good times play No worries on my mind, just a smile on my face I'm feeling so alive, in this happy place

    • Verse 2:
      Cool breeze on my skin, I feel so free The world is wide open, just waiting for me I'll dance in the sunshine, I'll sing in the rain I'll live every moment, and seize every day

    • Chorus:
      Nothing can bring me down, I'm feeling so high Summer is here, and it's time to fly I'll laugh and I'll dance, I'll live my best life Nothing can stop me now, I'm feeling so alive
Dialogue Explained
The songwriter is surprised at how well their writing partner (ChatGPT) captured the hopeful and inspirational tone of Williams’ “Happy.” They note the rhyme choices and scheme are a little inconsistent. Still, aside from the odd musical-inspired line about, ‘singing in the rain,’ it’s not terrible for a first try. The songwriter thinks they are now at the point where they can sit down, rework the lyrics themself, and write the remaining verses and bridge as a human.
Key to the success of this conversational interaction with the chatbot is the appearance that it ‘remembers’ previous bits of the conversation when, in fact, the AI does not actually remember the conversation but has only a limited ability to inspect and use prior data as input. The above limited scenario highlights how ChatGPT, when used as a dialogic partner in this way by its human user, is capable of a constructive, iterative conversation that yields both useful outputs and user learning (and resulting action). In dialogue with their AI writing partner, the songwriter uses their prompt requests to try out adjustments in style, language, and tone. The AI chatbot provides numerous examples for our ‘stuck’ human user’s consideration. The human songwriter can then compare the output examples, consider them in the context of their desired song topic and affiliated themes, and learn to recognize ChatGPT’s inconsistencies and information gaps. The final task, however, remains the work of the human songwriter: to write and complete their song and improve upon the AI output.

Quick Summary: What can and can’t ChatGPT do?

What it can do:
  • ChatGPT produces reasonable summaries of knowledge.
  • ChatGPT is able to reproduce and integrate academic arguments if the particular subject area and topic are part of its data pool.
  • ChatGPT can compare and contrast texts, theories, and approaches.
  • Its ability to accurately identify and provide legitimate sources and citations is hit and miss. Be careful. Watch for signs of fabrications (factual inaccuracies), also called 'hallucinations' in common AI-industry parlance, in the LLM's generated outputs.
  • ChatGPT responds to additional information. For example, to user requests for shorter or longer responses, or adjustments to style, tone, and reading level.
  • The quality of its outputs can surprise.
  • Trained on a wide variety of texts, ChatGPT is able to produce text outputs that cover a wide breadth of knowledge domains, cultural topics, and languages
    • ChatGPT can understand natural language prompts in many languages and can translate between them ( English > Japanese, Japanese > German, and so on)
What it cannot do:
  • Be trusted at face value.
    • It is essential that instructors and students carefully validate accuracy of generated output because…
      • It may hallucinate sources and citations
      • Generated outputs may contain factual inaccuracies
      • It may duplicate biases found in its training data
      • It is prone to mistakes in logic and mathematics.

See U of A’s Dr. Ali Shiri’s article, “ChatGPT and Academic Integrity,” for a quick rundown of the diverse kinds of “textual artifacts” the tool is able to generate. (And for more on ChatGPT’s limitations, see OpenAI’s list below - OpenAI, ChatGPT Limitations.)

Limitations
ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows. ChatGPT is sensitive to tweaks to the input phrasing or attempting the same prompt multiple times. For example, given one phrasing of a question, the model can claim to not know the answer, but given a slight rephrase, can answer correctly. The model is often excessively verbose and overuses certain phrases, such as restating that it’s a language model trained by OpenAI. These issues arise from biases in the training data (trainers prefer longer answers that look more comprehensive) and well-known over-optimization issues. 1, 2 Ideally, the model would ask clarifying questions when the user provided an ambiguous query. Instead, our current models usually guess what the user intended. While we’ve made efforts to make the model refuse inappropriate requests, it will sometimes respond to harmful instructions or exhibit biased behavior. We’re using the Moderation API to warn or block certain types of unsafe content, but we expect it to have some false negatives and positives for now. We’re eager to collect user feedback to aid our ongoing work to improve this system.

As for ChatGPT’s use as an AI tool in teaching and learning applications, we also suggest you carefully read through OpenAI’s documentation on this subject:

  1. Considerations for Educators
  2. ChatGPT General FAQ

CAUTIONS and Considerations

Ethics – Privacy and Data Security

In any educational setting, especially when utilizing third-party software not officially supported by an institution like the University of Alberta, it is crucial to consider privacy, security, and student and instructor intellectual property. These concerns are particularly relevant when implementing Generative AI tools, such as ChatGPT, for academic purposes.

Generative AI tools like ChatGPT collect and process substantial amounts of personal data, which raises concerns about privacy and data security (for both students and instructors). For an understanding of industry practices in this area, one can refer to OpenAI’s Privacy Policy and Terms of Use. Notably, OpenAI actively collects and uses user data to enhance the platform and also limits its usage to individuals who are 18 years of age or older.

With respect to data storage, although ChatGPT recently introduced an option for users to delete previous conversations, this data (called “user conversation history”) remains in OpenAI data collection system for platform improvement purposes. Consequently, there exists a risk that such data could be exploited for malicious purposes and/or be compromised during a data breach.

As the result of just such a breach in late March, 2023 (see OpenAI post), Italy’s data protection agency began an investigation into OpenAI and banned ChatGPT (McCallum, 2023). This raises the question of whether other nations will follow suit and how this might impact other AI-tools as well. In early April, 2023, Canada’s federal privacy commissioner launched an investigation into Open-AI’s after receiving a complaint alleging privacy and data security concerns (Fraser, CBC News).

 

Alternative options: If you decide to incorporate AI-tools into your course, one potential solution for mitigating privacy concerns is to suggest that students use the program through anonymous email accounts; however, this does not prevent OpenAI from collecting other personal information such as IP address, browser type and settings, user's device, and user interactions on their site.

Also, keep in mind that some students (or even instructors) may still be unwilling to create a profile or share their user data in this way. Plan to offer students the option to opt-out of AI-tool based activities and prepare alternative assessments for those who choose not to participate. Other options to consider include the following: ask such students to submit example prompts (demonstrating what’ inputs’ they might give the AI-tool if they were using it), have them provide their own examples for class discussions, or pair them with peers (using the AI-tools) in group assignments.

Ethics - Accessibility and Authorship

Accessibility - Cost: While, currently, numerous AI tools are available for free, this situation may very well change in the near future. If you decide to integrate these tools into your course, ensure that you try to choose options that are accessible to all students. As much as possible, avoid AI-using tasks that may disproportionately favour those students who can most afford the associated costs. We need only look at the latest version of OpenAI's GPT model. GPT-4, which is accessible only through a subscription to ChatGPT Plus (current cost: $20.00 USD per month) to see an example of the business model shift from ‘free’ to ‘for-pay’.

Accessibility - UDL: Recognize that not all AI tools are designed to be universally accessible. This means instructors, when selecting tools for use by their students, should consider accessibility concerns for students with different abilities, disabilities, and identified and undisclosed needs for success.

Authorship: AI tools like ChatGPT generate text outputs based on the extensive data lakes (data pools or sets) they have been trained on and in response to a human-user’s input (prompt). This also can make it challenging to, with 100% certainty, ascertain responsibility for the content, determine the author, and establish accountability for the created material. When viewed from a certain perspective, this new human-AI partnership raises questions about whether the student (the prompt writer) or the AI tool (output generator) should be credited for the work produced. However, at the University of Alberta, we frame such questions in the light of the U of A’s (September 2024) Student Academic Integrity Policy which explicitly references Artificial Intelligence and its misuse for academic advantage. Instructors need to make explicit to students that submitting work generated by AI without providing appropriate attribution (and or citation) could be considered cheating.

Problems to Note - Biases and Fabrications

Biases: Explain to your students the limitations and biases that are sometimes found in AI-generated content. Encourage them to use these tools responsibly and with critical reflection. AI tools are only as biased as the data on which they are trained. That means that the biases within the data used to train the AI will also appear in the generated results. As a result, AI tools may perpetuate biases present in their original training sets, leading to discrimination against certain groups and the reinforcement of stereotypes and existing inequalities.

Much research over the past twenty years has explored bias in search engines like Google and Microsoft's Bard, whether stemming from data sources or AI-driven algorithms. Algorithms, built by biased individuals (with non-neutral corporate interests), can reinforce systemic racism and other biases in search results (Noble, 2018). Moving forward, it will be equally vital to scrutinize the operational and technological parameters of LLMs like ChatGPT and GANs like Midjourney in order to mitigate the risks for biases while examining the ethical implications of the consequences from biased outputs (Ferrara, 2023).

Fabrications (Factual Inaccuracies): Certain AI-tools, for instance, Large Language Models like ChatGPT, are prone to ‘fabrications.’ This means their generated output may contain inaccurate, misleading, and/or harmful content, which potentially creates or perpetuates falsehoods based on their training data. The AI model is designed to sound convincing; thus, its primary concern is that its output appears factual. This is very different than if the platforms (or apps) were designed to generate only true statements. For this reason, human users of this kind of AI tool need to be vigilant by watching for generated output that may seem real but is clearly false (when scrutinized by knowledgeable individuals and experts, and/or when critically reflected on by instructors, students, and learners).

In the context of teaching and learning, the emergence of AI is the chance to rethink our focus on written products and instead focus on what we as instructors are having our students do to develop higher-order critical thinking skills. It is in this light that ChatGPT can be employed to encourage improved student comprehension, critical inquiry and analysis, and metacognition. Challenge them to identify and explicate inaccuracies, omissions, and unaddressed aspects of the AI-tool’s outputs. Leveraging the convincing tone of the AI's outputs, educators can support students to not only critically examine AI-generated content but also accept the academic challenge of correcting and improving it.

AI Literacy - A Key Consideration

As artificial intelligence becomes increasingly integrated into our daily lives and educational systems, developing AI literacy will be essential for both instructors and students. For instructors, understanding (and playing with) AI should enable us to, where appropriate and applicable, better use AI-tools in our teaching to improve the overall learning experience for students. By learning about AI's capabilities, limitations, and ethical concerns, instructors can make informed decisions on how, if they should choose, to incorporate AI into their courses.

AI literacy is equally as important for students. It is incumbent on us to help prepare them to successfully navigate a world where AI technologies will likely play a significant role. Becoming AI literate will support students’ efforts to critically engage with AI systems’ outputs, be aware of potential biases, and develop essential critical thinking and problem-solving skills. The promotion of AI literacy across the curriculum could very well empower students with the knowledges and skills they need to excel in their careers and contribute meaningfully to the ever-evolving world of technology (Southworth et al).

In short, it is a likely outcome that both instructors and students will benefit from cultivating AI literacy because AI literacy will prepare us all for the challenges and opportunities presented by the expanding presence of AI in education and various other domains.

generative-ai-gpt-4.png

What is GPT-4?

In mid-March of 2023, OpenAI released its most powerful language model yet: GPT-4, which they claim vastly outperforms its predecessor, ChatGPT (GPT-3.5), released only four months prior (Open AI, GPT-4). According to OpenAI, “The difference comes out when the complexity of the task reaches a sufficient threshold—GPT-4 is more reliable, creative, and able to handle much more nuanced instructions than GPT-3.5” (OpenAI, GPT-4).

It appears that GPT-4 is much better than its predecessor at standardized test taking as well.

Simulated Exams

For more on ChatGPT’s Simulated Exams, see OpenAI’s reseach below (OpenAI, ChatGPT Research).

Alt text: A table comparing the performance between GPT-4, GPT-4 with no vision, and GPT 3.5 on popular standardized test such as the Bar exam.

Simulated exams

GPT-4

estimated percentile

GPT-4 (no vision)

estimated percentile

GPT-3.5

estimated percentile

Uniform Bar Exam (MBE+MEE+MPT)1

298 / 400

~90th

298 / 400

~90th

213 / 400

~10th

LSAT

163

~88th

161

~83rd

149

~40th

SAT Evidence-Based Reading & Writing

710 / 800

~93rd

710 / 800

~93rd

670 / 800

~87th

SAT Math

700 / 800

~89th

690 / 800

~89th

590 / 800

~70th

Graduate Record Examination (GRE) Quantitative

163 / 170

~80th

157 / 170

~62nd

147 / 170

~25th

Graduate Record Examination (GRE) Verbal

169 / 170

~99th

165 / 170

~96th

154 / 170

~63rd

Graduate Record Examination (GRE) Writing

4 / 6

~54th

4 / 6

~54th

4 / 6

~54th

USABO Semifinal Exam 2020

87 / 150

99th–100th

87 / 150

99th–100th

43 / 150

31st–33rd

USNCO Local Section Exam 2022

36 / 60

38 / 60

24 / 60

Medical Knowledge Self-Assessment Program

75%

75%

53%

Codeforces Rating

392

below 5th

392

below 5th

260

below 5th

AP Art History

5

86th–100th

5

86th–100th

5

86th–100th

AP Biology

5

85th–100th

5

85th–100th

4

62nd–85th

AP Calculus BC

4

43rd–59th

4

43rd–59th

1

0th-7th

 

GPT-4 can pass the bar exam in the 90th percentile, while the previous model struggled, sitting in the 10th percentile. GPT-4 scored in the 88th percentile on the LSAT, up from GPT-3.5’s 40th percentile. Its improvement in SAT performance is also noteworthy in both the Evidence-Based Reading and Writing and Math exams. And on the Medical Knowledge Self-Assessment Program test, GPT-4 performed much better, with a score of 75% percent (GPT3.5 sat at a lowly 53 percent).

These are impressive results, not just in terms of how well the model can demonstrably perform, but also in terms of the rapid pace of the AI’s progress and its adoption through partnerships with developers and businesses in any number of areas: language translation and learning (Duolingo), image description, fraud detection, and education (Khan Academy).

Another claim OpenAI proudly makes is that GPT-4 “...can generate, edit, and iterate with users on creative and technical writing tasks, such as composing songs, writing screenplays, or learning a user’s writing style.” To demonstrate GPT-4’s capabilities in this vein, let's revisit our Pharrell-inspired summer song and task the AI with a slightly more difficult series of prompts.

Prompt #4 (#1 GPT-4)

  • Human User
    Let's get on to the next step in our exercise. The song we've created is timeless and inspirational. I want to keep the inspirational feeling but introduce a new element to the song: romance. I love the futuristic romance-inspired songwriting of Janelle Monáe on their 2010 album release, "ArchAndroid."

    Here's an example of lyrics from their song, “Say You’ll Go”:

    • [Verse 1]
      Love is such a novelty
      A rarely painted masterpiece
      A place few people go or ever know
      An underwater rocket love
      Exactly what I'm searching for
      If you're brave enough to go
      Tell me so…

    • [Chorus] Say you'll go to nirvana
      Will you leave Samsara? I
      n the words of Dhammapada
      Who will lead? Who will follow?
      Our love will sail in this ark
      The world could end outside our window
      Let's find forever
      And write our name in fire on each other's hearts

    • [Verse 2]
      Love is not a fantasy
      A haiku written in Japanese
      A word too often used but not believed
      Witness the interaction of
      The flood, the sea, the sky, the dove
      Time erodes the shore but not our love

    • [Chorus]

    • [Outro]
      Do do do
      Do, do, do, do
      The world could end today...
      Do, do, do, do, do
      Our love will sail in this ark
      The world could end outside our window
      We'll find forever
      Oooooo...
      Love is not a fantasy
      It's more for you, you and me


    Can you compare and contrast the Janelle Monáe song with the Pharrell-inspired summer song lyrics? What important considerations do I, as a songwriter, need to keep in mind as I attempt to combine the two themes: summer (as season of hope) and romance (romantic love)?

    Also, if you can provide a brief overview of the importance of these two themes in popular songwriting (with examples) of the last fifty years that would be helpful.

  • GPT-4
    The Janelle Monáe song and the Pharrell-inspired summer song have distinct thematic differences, while both maintain an uplifting and positive tone. The Pharrell-inspired lyrics primarily focus on the joys of summer and the feeling of freedom, with no direct reference to romance. Conversely, the Janelle Monáe song delves into the complexity of romantic love, using poetic language and vivid imagery to describe its beauty and depth.

    When attempting to combine the themes of summer (as a season of hope) and romance (romantic love), it is crucial to find a balance between the two. You can use summer as a backdrop for romantic encounters, highlighting the warmth and vibrancy that the season brings to a blossoming relationship. Intertwine the emotions of hope and love, utilizing the seasonal atmosphere to enhance the romantic narrative.

    The themes of summer and romance have been prevalent in popular songwriting over the last fifty years. Examples include:

    • "Summer Nights" by Olivia Newton-John and John Travolta (1978) - This song from the musical "Grease" narrates a summer romance between the main characters.
    • "Summer of '69" by Bryan Adams (1984) - The song reminisces about the summer of 1969, touching on themes of nostalgia, youth, and love.
    • "(I've Had) The Time of My Life" by Bill Medley and Jennifer Warnes (1987) - Featured in the movie "Dirty Dancing," this song encapsulates the passion and romance of a life-changing summer.
    • "The Way You Look Tonight" by Elton John (1994) - A romantic ballad that captures the feeling of being in love during the summer.


    These examples demonstrate the versatility of combining summer and romance themes, creating songs that resonate with audiences and evoke strong emotions.

Dialogue Explained
Since the above comparison, generated in mere seconds, is a fairly simple task for the two models, a side-by-side comparison of ChatGPT and GPT-4 text outputs might yield only subtle textual differences. However, if followed with ever more complex prompts, the two models’ outputs would begin to become easily distinguishable. GPT-4’s capabilities far exceed its predecessor in countless ways. That said, GPT-4, like its predecessor, is still prone to conveying biases that may be present in its data lakes and training protocols as it generates text based on its complex algorithmic models. See the next Prompt.

Prompt #5 (#2 GPT-4)

  • Human User
    Thanks for the brief summary. The idea of using summer as a backdrop might allow us to play with the other seasons to emphasize and contrast feelings associated with the loss of love.

    I know I asked for musical examples from the last 50 years, but the songs and artists you provide as examples are definitely dated and, some would say, biased. Can you suggest other examples? What about artists who are *not white?

  • GPT-4
    I apologize for any perceived bias in my previous examples. Here are more diverse examples of songs from various time periods and artists that incorporate themes of summer and romance, showcasing different cultural and musical perspectives:

    • "Summertime" by DJ Jazzy Jeff & The Fresh Prince (1991) - Will Smith, an African American artist, raps about the joys of summer, including love and good times with friends and family.
    • "Cruel Summer" by Bananarama (1983) - This British girl group tells the story of a summer romance that turns bittersweet as the season ends.
    • "Rockaway Beach" by Ramones (1977) - A punk rock classic that celebrates the beach, sun, and the possibilities of summer romance.
    • "Summertime" by Ella Fitzgerald & Louis Armstrong (1957) - This jazz standard, performed by two African American legends, has a slower, more melancholic tone, evoking the longing for love during the summertime.
    • "La Bicicleta" by Carlos Vives & Shakira (2016) - This upbeat Latin pop song, sung in Spanish by two Colombian artists, celebrates the simple pleasures of life, love, and summertime.
    • "Summer Love" by Justin Timberlake (2007) - This modern pop song tells the story of a fleeting summer romance that both partners wish could last forever.


    These examples highlight the diversity of artists and styles that have explored the themes of summer and romance in their music, allowing you to draw inspiration from various sources while creating your own unique composition.

Dialogue Explained
GPT-4 receives its criticism and new guidance. It generates more nuanced and potentially more on-target outputs as outlined in the human user’s prompt(s): see its attempt to provide the songwriter with a somewhat more *diverse (but shallowly diverse– The Ramones and Justin Timberlake? But also, perpetuating romance and love heteronormatively and binarily) set of historical songwriting precedents.

(*GPT-4’s analysis of the prompt led it to generate output that, while indeed more diverse in terms of its selection of musical genres and racialized identities of musicians, is hardly representative in the spirit of the request. Again, this is another example of how biases are found to be present in the tool’s generated outputs. 
This is quite possibly an example of the biases present in the data being ‘harmlessly’ regenerated as part of the chatbot’s output. Human users must remain vigilant. In a teaching and learning context, output such as this would warrant, at minimum, a discussion about the dangers of algorithmic and data bias in AI-tools generated outputs. The discussion might also include strategies for how to detect and mitigate against the risks associated with biased content and expression.

Let’s now pick up the songwriter’s brainstorming session in progress:

Prompt #6 (#3 GPT-4)

  • Human User
    Well, you can STILL do a lot better in terms of representation, but we’ll save that work for next time.

    Let's now return to our songwriting experiment. How about we draw inspiration from both our Pharrell Williams' summer lyrics and Janelle Monáe's romantic lyrics? Let's create a new song mash-up.

  • GPT-4
    Certainly! Here's a new song that combines the upbeat and positive energy of the Pharrell-inspired summer lyrics with the romantic and poetic elements from Janelle Monáe's lyrics:

    • Verse 1: The sun is shining bright, a love story unfolds Our hearts ignite, as the summer takes hold No worries on our minds, just the warmth we embrace Together we'll find, our hearts' happy place

    • Verse 2: A soft breeze on our skin, we feel so alive The world is our canvas, as we dance through the night Hand in hand we'll journey, exploring love's sweet terrain In sunshine and moonlight, our love will remain

    • Chorus: Nothing can bring us down, our love will take flight Summer's embrace, and a love burning bright We'll laugh, dance, and cherish, the moments we share In the magic of summer, we'll find love everywhere

    • Bridge: Our love's a masterpiece, painted in fiery hues A story that unfolds, with each day anew In the warmth of the sun, and the touch of the sea We'll write our own tale, of love and destiny

    • Chorus: Nothing can bring us down, our love will take flight Summer's embrace, and a love burning bright We'll laugh, dance, and cherish, the moments we share In the magic of summer, we'll find love everywhere

     

Dialogue Discussion
Although the output above perhaps leans a bit too much towards the Pharrell Williams’ end of the songwriting spectrum, the songwriter is given a set of lyric options from which they can further contemplate and explore their ideas of a Williams-Monáe mashup. But it doesn’t have to stop there. This also provides an opportunity to illustrate how GPT-4 can further iterate (respond to and) with human users for creative and productive purposes.

Prompt #7 (#4 GPT-4)

  • Human User
    Can you explain your lyric choices? I feel you may have lost some of the poetic allusiveness of Monáe's lyrics.

  • GPT-4
    I apologize if the new song did not fully capture the poetic allusiveness of Janelle Monáe's lyrics. In creating the mash-up, I aimed to combine the upbeat and positive energy of the Pharrell-inspired summer lyrics with the romantic elements from Monáe's lyrics, while maintaining a balance between the two styles.


    However, I understand that some of the poetic depth from Monáe's lyrics might have been lost in the process. In the new song, I primarily focused on capturing the essence of summer and romance, using imagery such as sunshine, dancing, and the sea to evoke a sense of warmth and love.

    To better incorporate the poetic allusiveness of Monáe's lyrics, I suggest revising the song with more vivid imagery and metaphorical language that explores the depth of romantic love. Here's an attempt at revising the first verse and chorus to better reflect Monáe's style:

    • Verse 1: In the sun's warm embrace, a love tale begins Our hearts intertwine, as the daylight dims A celestial dance, our fates interlaced In the summer's sweet rhythm, our love finds its pace

    • Chorus: Through the infinite cosmos, our love takes its flight In the summer's soft glow, we'll embrace the night We'll sail through the stars, our journey declared In the vastness of space, we'll find love prepared


    Please let me know if you would like further revisions or if there are specific elements from Monáe's lyrics that you would like to see more prominently featured in the song.

Dialogue Explained
There the songwriter has it, the beginnings of their Williams and Monáe hybrid mash-up, part inspirational summer song, part chivalric space romance. Now, all that’s left for them to do is the hard part, to find the time and energy they will need to properly rewrite the lyrics, set it to music, and record their song.
Now, what if this songwriting experiment was part of a class-based assignment? What if the songwriter needs some help summarizing the good work they’ve been doing for a brief oral presentation in class? Can GPT-4 help them with this task?

Prompt #8 (#5 GPT-4)

  • Human User
    That's a great start. Thank you. The rest of the work is mine. One last task. I have to present a short summary of my song project in class tomorrow. Any suggestions?

  • GPT-4
    Certainly! Here's a brief summary of your song project for your presentation:

    • In this songwriting project, we aimed to create a unique composition that combines the uplifting and hopeful theme of summer, inspired by Pharrell Williams, with the poetic and romantic elements of Janelle Monáe's lyrics. Drawing from diverse influences, we crafted a song that explores the depth and beauty of romantic love set against the backdrop of a magical summer. The lyrics utilize vivid imagery, metaphorical language, and a sense of wonder to evoke strong emotions and connect with the listener. This fusion of styles highlights the versatility of songwriting and the power of music to express the complexities of human experiences, transcending boundaries and genres


    Feel free to modify this summary as needed to better fit your presentation, and good luck with your class!

Dialogue Explained
The above summary is accurate. It captures much of the subject and emotional matter that drives the intent behind the songwriter's mash-up.

Still, since the songwriter’s memory is far from perfect, what if they shortened the summary even further? What about a single sentence?

Prompt #9 (#6 GPT-4)

  • Human User
    Thank you. Oh, did I mention I can't read my presentation. I have a horrible memory. Can you provide a single sentence summary? Maybe this will help me to remember.

  • GPT-4
    Of course! Here's a single sentence summary for your presentation: "In this songwriting project, we merged the upbeat summer theme inspired by Pharrell Williams with the poetic romanticism of Janelle Monáe's lyrics, creating a unique composition that celebrates the magic of love in the enchanting setting of a sunlit summer."

And what if the song writer loves using alliteration as a memory aid? Can GPT-4 get them there?


Prompt #10 (#7 GPT-4)

  • Human User
    That is still a lot for me to remember. Say, I love alliteration. Any chance you can create a single sentence summary? A sentence that begins with "C."

  • GPT-4
    Certainly! Here us a single-sentence summary with alliteration:

    C: "Combining captivating summer cheer with the charm of romantic connections, this composition conveys the enchantment of love in a sun-soaked season."

Dialogue Explained
"Combining captivating summer cheer with the charm of romantic connections, this composition conveys the enchantment of love in a sun-soaked season." The answer to the songwriter’s question, “Can GPT-4 get them there?”, is most definitely, a yes.
The above series of summary-oriented prompts and outputs demonstrates the tremendous summarizing capabilities of GPT-4. It is better able to follow the directions and intentions of its human user than is its predecessor (and search engine AI peers, Bing, and Bard). This ability could aid students who might seek to learn key concepts and theories using GPT-4 as a tool to produce study summaries and simplified explanations of difficult subject matter. However, as the outputs above prove, the user must also be ready to interrogate and critically assess the AI-tool’s outputs for signs of biases and fabrications.

Quick Summary: GPT-4 - What can and can’t it do?

What it can do:
  • GPT-4 is multimodal, the human user is now able to input natural language prompts and images (a major step forward).
  • Remains unimodal, text-based outputs.
  • If a user inputs images, GPT-4 can generate descriptions, classifications, and visual analyses.
  • Human-users can include larger amounts of token (text and data) into task prompts than previously possible.
  • Greater analytic abilities of structured data sets.
  • It can still compare and contrast texts, theories, and approaches. Ability to generate factual information (vs. fabrication) is much improved.
  • Its ability to accurately identify and provide legitimate sources and citations is much more accurate. Need to still watch for fabrications – fictional articles and authors may still appear. (However, this is a big step up.)
  • Responds more accurately to additional information. For example, to user requests for specific adjustments to style, tone, and textual contents. More ‘creative’ in this respect.
  • The quality and depth of its outputs will surprise.
  • Increased multilingual language abilities.
  • Greater coding and mathematical capabilities.
What it cannot do:
  • Be trusted at face value.
    • Essential that instructor and student carefully validate accuracy of generated output.
      • May fabricate the odd source and citation.
      • Generated output may contain factual inaccuracies.
      • Will replicate and perpetuate biases found in its training data.

Conclusion (for Now)

Of particular interest to educators will likely be Microsoft’s GPT-powered chatbot, Bing, and Google’s LaMDA-powered Bard. Though the text-based capabilities of the two different LLMs behind the search engines’ interactions with human users are largely similar, the ability of both Bing and Bard to actively search, access, and incorporate current information and data is a boon for those who wish to conduct online research.

In time (and with greater training), these tools will likely become more reliable with respect to sourcing and citing information more accurately. As well, these tools may be able to better combat LLM fabricate by legitimately accessing useful and relevant resources (including proprietary information sources) in real time.

* Of interest, please watch for signs of the various guardrails both Google and Microsoft have put in place to try to ensure that their chatbots are not used for malicious or illegitimate purposes.

For a quick summary overview of these AI-tools, please see: Chatbots Compared (Vincent et al)


Like the Internet, email, and perhaps even the calculator, AI is a technological leap. Moving mail instantly through electronic sources must have seemed magical to a particular generation. If you think about it, having email is rather astounding, but we are no longer astonished by or afraid of its existence. Rather, we’ve learned to integrate email into our daily communication. As multifaceted work and entertainment centres that can also help us out with GPS, a smartphone is also rather extraordinary. However, the once seemingly science-fictional technology, in time, becomes more ordinary than we may have initially imagined or feared. You should acquaint yourself with a Generative AI tool or tools. Try them, so you can see for yourself what it can and cannot do. Most likely, doing so will enable you to quell your fears and exercise your imagination. It’s not exactly a robot or android as science-fiction may have envisioned, but you have a potentially powerful and helpful tool at your ready.

Sources

Elliot, Lance. ChatGPT As A Platform Gets Bigger And Bolder As OpenAI Rolls Out Plugins For All Kinds Of Add-On Apps, Stewing Up AI Ethics And AI Law (2023, March 25) Forbes https://www.forbes.com/sites/lanceeliot/2023/03/25/chatgpt-as-a-platform-gets-bigger-and-bolder-as-openai-rolls-out-plugins-for-all-kinds-of-add-on-apps-stewing-up-ai-ethics-and-ai-law/?sh=4c0b94d0446d

Ferrara, Emelia. (2023, April 7) Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models arXiv: 2304.03738 https://doi.org/10.48550/arXiv.2304.03738

Floridi, Luciano. AI as Agency without Intelligence: On ChatGPT, Large Language Models, and Other Generative Models (2023, Feb 16) Teaching and Philosophy 2023 SSRN: https://ssrn.com/abstract=4358789 or http://dx.doi.org/10.2139/ssrn.4358789

Fraser, David. (2023, April 4) Federal privacy watchdog probing OpenAI, ChatGPT following complaint CBC News https://www.cbc.ca/news/politics/privacy-commissioner-investigation-openai-chatgpt-1.6801296

[Video 10:29] Fyshe, Alona. Does AI actually understand us? TED(@DestinationCanada) https://www.ted.com/talks/alona_fyshe_does_ai_actually_understand_us

McCallum, Shiona. ChatGPT banned in Italy over privacy concerns (2023, April 1) https://www.bbc.com/news/technology-65139406 Open AI. March 20 ChatGPT Outage: Here What Happened (2023, March 24) https://openai.com/blog/march-20-chatgpt-outage

Noble, S. (2018). Algorithms of Oppression: How Search Engines Reinforce Racism. New York, USA: New York University Press.

Open AI. March 20 ChatGPT Outage: Here What Happened (2023, March 24) https://openai.com/blog/march-20-chatgpt-outage

OpenAI – See in-text links.

Shiri, Ali. (2023, Feb 2) ChatGPT and Academic Integrity February. Information Matters https://informationmatters.org/2023/02/chatgpt-and-academic-integrity/

Southworth et al. (2023) Developing a model for AI Across the curriculum: Transforming the higher education landscape via innovation in AI literacy. Computers and Education: Artificial intelligence (4) https://ai.ufl.edu/media/aiufledu/resources/Developing-a-Model-for-AI-Across-the-Curriculum-Transforming-the-higher-ed-landscape-via-innovation-in-AI-literacy.pdf

Vincent, James, et al. AI (2023, Mar 24) Chatbots compared: Bard vs. Bing vs. ChatGPT. The Verge https://www.theverge.com/2023/3/24/23653377/ai-chatbots-comparison-bard-bing-chatgpt-gpt-4

Wolfram, Stephen. What is ChatGPT Doing…and Why Does it Work? (2023, Feb 14) Writings https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/

Suggested resources for further learning:

AI Impacts 2022 Expert Survey on Progress in AI August 3, 2022 AI Impacts https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/


Generative AI (and LLMs)

Gozalo-Brizuela, Roberto, and Garrido-Merchan, Eduardo. (2023, January 11) ChatGPT is not all you need. A State of the Art Review of Large Generative AI models Arxiv(.org). Cornell University https://doi.org/10.48550/arXiv.2301.04655

Stokel-Walker, C., & Van Noorden, R. (2023). What ChatGPT and generative AI mean for science. Nature, 614(7947), 214-216–216. https://doi-org.login.ezproxy.library.ualberta.ca/10.1038/d41586-023-00340-6

ChatGPT (GPT-4)

Wolfram, Stephen. What is ChatGPT Doing…and Why Does it Work? (2023, Feb 14) Writings https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/

[Video 10:00] Mushoff, Till. But How Does ChatGPT Actually Work? (An instructional video) YouTube

[Video 16:03] Beyond ChatGPT: what chatbots mean for future (2023, March 23) The Economist Youtube https://www.youtube.com/watch?v=dctcfxw13AQ