(January 2024) AI Uses in Teaching: Session 3 of AI in Education Series

0:00
[Casey Rinehart] I think we are at 11:01, so we can we can get started.

0:06
Hello everyone, and welcome to session 3 in AI in

0:09
Education series brought to you by EHE’s Center for Digital Learning and

0:14
Innovation.

0:16
If we can just go to the next slide there, Ivan, then I’ll get us introduced.

0:21
So first, I’d like to introduce and welcome our

0:24
expert host for today’s session, Doctor Peter Sayer and Dr.

0:27
Ivan Stefano.

0:29
Dr.

0:30
Sayer is a professor of multilingual language education and EHE’s Department

0:34
of Teaching and Learning.

0:36
His work is in applied linguistics and educational sociolinguistics with a focus

0:42
on language learners and bilingual and ESOL programs.

0:47
And Doctor Stefano is the Director of Intercultural English Language Programs,

0:52
or IELP, and in EHE’s Department of Teaching and

0:56
Learning.

0:57
His work focuses on helping international and multilingual students, scholars,

1:01
and professionals with their English proficiency and cultural adjustments in

1:06
the United States.

1:07
In the summer of 2023, they taught a course on special topics in

1:12
education, including a topic on AI and the future of

1:16
digital tools and language teaching.

1:19
And so because of that, we’re just incredibly thrilled that they

1:23
agreed to host this session for us and and share their knowledge and insights

1:28
with all of us around AI uses and teaching.

1:31
So without further delay, I will turn it over to Doctor Stefano and

1:36
Doctor Sayer.

1:40
[Dr. Stefano] Hi, everyone.

1:41
Good morning and thank you, Casey, for the introduction.

1:45
So, so Dr. Sayer and I taught this course for the

1:50
summer.

1:51
So we’re going to try to pack it in, in about 25 minutes today and then

1:55
questions and answers and discussions after that.

1:58
So we’re going to focus on a few things, but hopefully what we cover today can be

2:06
used in your context after the session today.

2:11
All right, so I want to start with how Gen.

2:15
AI works.

2:15
How many of you are familiar or have used ChatGPT or other AI tools before?

2:25
OK, so, so as you know, ChatGPT or, or AI is a large language model or LLMS,

2:34
which is basically a computer program with a lot of data,

2:40
a lot of examples that can recognize human language.

2:47
And then the text and the data will be presented to you so that it makes sense

2:55
and it use analysis and things like that to organize the the order of the words.

3:03
And it also continues to be fine-tuned as we go.

3:09
And for now, AI, a lot of AI programs can also do more

3:14
than just text, you know, images, videos, music and voices.

3:19
And there’s, there’s a lot of AI products right out

3:23
there that can actually combine all these things.

3:27
So that’s just a quick overview.

3:29
And in this session, we’re, we’re going to talk about ChatGPT and

3:34
other AI tools that we have been exploring.

3:38
So I’m going to turn it over to Dr. Sayer.

3:41
So we’re going to talk about prom engineering here,

3:44
which is what we use when we use AI products.

3:48
[Dr. Sayer] Great, Thanks Ivan.

3:49
Good morning everyone.

3:50
So yeah, as Ivan said, I think we’re going to go through this

3:53
pretty quickly.

3:54
I think we’ve got about 25 minutes, but then we’ll leave plenty of time for

3:57
for discussion.

3:58
So if I, if I go over it too quickly, we can certainly come back to it.

4:02
But what we found is and you know everyone,

4:05
most everyone said they’ve they’ve used ChatGPT and you know that it’s,

4:08
it’s pretty intuitive, right?

4:10
Basically any, any command or question that you put into

4:13
the, into the, the, the chat box there, the prompt, we’ll, we’ll generate,

4:18
you know, an output from ChatGPT.

4:21
But what we found in the class and when we were sort of when,

4:24
when our students were kind of pushing it to try to do different things.

4:27
And a lot of our students were teachers.

4:28
And so they were trying to, you know, figure out how do I get it to help me

4:32
develop materials for my teaching or, or rubrics and things like that.

4:36
And, and a lot of it, ChatGPT did really well,

4:38
but in some parts where they were finding that the,

4:41
the output that they were getting from ChatGPT just wasn’t really working.

4:46
And they thought, well, ChatGPT, it can’t do this, right.

4:48
And it turned out usually it wasn’t a matter of the limitation of ChatGPT.

4:53
It was a limitation on like us understanding how do we generate the like

4:57
high quality output from ChatGPT.

4:59
So it really was a, a question of what we call prompt

5:02
engineering.

5:03
So essentially what, what we’re trying to do here,

5:06
prompt engineering or what we call prompt engineering patterns are really just the

5:11
kind of like strategy that you’re using when you’re putting that, that, that,

5:16
that instruction into the chat bot into that chat box for ChatGPT or whatever AI

5:20
you’re using.

5:21
It’s really just that strategy to get the AI to understand what you’re looking for.

5:26
So there’s, there’s a whole bunch of these.

5:28
And as you, as you get into it, I actually did a MOOC on this.

5:33
Coursera has a really good course on prompt engineering to do like the whole

5:37
course.

5:37
And they get kind of into the weeds on all of the various patterns.

5:41
So if you’re interested in going a lot further or, you know, getting some,

5:45
some expertise on, on specifically on how to use these

5:48
prompt engineering patterns, I recommend that Coursera MOOC.

5:52
But I just want to use the some that, that,

5:54
that I think are kind of like the most basic ones and like for us as,

5:58
as instructors, probably the most powerful.

6:00
So, so we’re going to use these.

6:03
I don’t know if the second one, I called it a pattern.

6:04
I don’t know if it’s really a pattern, but I’ll show you what it is too.

6:06
So we’ll, we’ll just kind of cover these, these basic ones.

6:09
And as I mentioned there, there’s there’s a lot more advanced stuff

6:12
that you can do with it kind of based on different combinations of these.

6:17
So thanks Rick for that, that Coursera link there.

6:20
I can go ahead and jump to the to the next one if you.

6:25
So I think the, the, the first pattern and maybe like the most

6:29
powerful one is what’s called the persona pattern.

6:33
And my, you know, when my, my son who’s was a 7th grader at the time

6:37
last spring when everybody was kind of discovering ChatGPT.

6:41
I remember when it, when you know, and I was starting to play around with it,

6:45
Ivan and I said, like, let’s, hey, let’s teach this course.

6:48
So we were kind of ourselves kind of doing a crash course on,

6:51
on trying to figure out what what all this was about and what it could do.

6:55
So, so my son’s, you know, doing his homework,

6:57
struggling on some book report that he had to do.

7:01
And so I said, well, what what you know, did you try ChatGPT?

7:04
What did it say?

7:05
And he’s like, well, and can I do that?

7:07
Am I allowed to do that?

7:07
And I’m like, well, you can’t turn it in, That’s cheating,

7:10
but you can see what it does, right?

7:12
And so, you know, he puts in this prompt, right, the review of the book, and,

7:16
you know, it spits out this beautiful book reviews

7:19
like, wow, this is like, this is really good.

7:21
I’m like, yeah, dude.

7:22
But that’s like way too good, right?

7:24
You wouldn’t actually write this, not in your voice.

7:26
Doesn’t sound anything like how you would write it.

7:28
And so one thing then he was like, yeah, yeah, you’re right.

7:32
So like, well, so then how would you get ChatGPT to

7:35
write a book review as a 7th grader?

7:37
Turns out ChatGPT is actually quite good at that.

7:39
You use the persona pattern.

7:40
Here’s the basic pattern here is you tell ChatGPT to act like person,

7:45
persona X to perform Y.

7:48
So basically, rather than saying write a review of the

7:50
book, which gives you kind of like the default

7:53
ChatGPT voice, you what you do is go to the next slide

7:56
item.

7:58
You just say that act as a 7th grader, Write a book.

8:01
In this case, it spit out something completely

8:03
different.

8:03
It changed, you know, syntactical complexity and lexical.

8:06
It changed everything.

8:08
It sounds a little odd, like, Oh my God, you guys show this book is so good.

8:12
It sounds like like kind of weird, right?

8:13
And so my son was like, no, that I wouldn’t write that either, though.

8:17
That sounds like really.

8:18
And so you started playing with it.

8:19
Go ahead and go to the next one, Ivan.

8:21
And what you’re doing here and what we’re calling iterative refinement,

8:25
it’s basically just like tweaking it, right?

8:27
You go through and tweak your prompt and we tweaked it over and over again until

8:31
he found that the output that he was getting kind of sounded and and was

8:35
organized kind of in the way that he wanted to.

8:38
And for me, this was kind of like an aha moment

8:40
because I thought of it as, you know, initially coming into it like I think

8:43
Benny did.

8:43
It’s like, well, students are going to use ChatGPT to

8:46
cheat, right?

8:47
Because you just put it in and it gives you the answer and then you use that.

8:50
But you know, when I, when I saw what, what he was doing here,

8:53
as he was trying to figure out how do I get ChatGPT to give me the kind of output

8:57
that I want?

8:58
And he was really analysing again, that’s sort of the linguistic decision

9:02
making that ChatGPT was doing.

9:04
He was really analysing that and kind of thinking about his own voice in writing

9:09
this.

9:09
And how do I, you know, understand if that really represents my

9:13
voice or not.

9:14
And so it really kind of changed the way I was thinking about using these sorts of

9:19
things as, you know, cheating or plagiarism versus as a

9:22
thinking tool, right?

9:24
And, and so seeing kind of his level of

9:25
engagement and how the assignment went from just like an, oh God,

9:29
I got to write this stupid book review to like, oh, this is like, really, oh,

9:33
and why, why did it choose that word and look at

9:35
how it changed it from, you know, each time he,

9:37
he tweaked the prompt and then regenerated the response, right.

9:41
And so it really was, you know, kind of illuminating for me to kind of

9:44
see see that process, see his kind of the gears turning as he

9:47
was like interacting with this and seeing what was happening.

9:51
So, so essentially that’s it, the persona pattern and then that that

9:55
tweaking it and the refinement, right?

9:58
And so eventually he kind of settled on a version of of that one there.

10:03
That was what what is it saying using straightforward language is is you know

10:08
what he came up with there.

10:10
So keep going, Ivan.

10:13
So again, that’s the, the, the, the basic formula for that pattern, right?

10:18
And so we tried to do this too.

10:19
And we were interested in like, well, could we, you know,

10:23
particularly where Ivan works, he works with a,

10:26
a lot of folks who are the instructors in our intensive English program,

10:30
our our intercultural English language program as ESL instructors.

10:34
And so we’re like, well, could we develop training materials for

10:37
the instructors to see like, OK, could we get ChatGPT to write things as

10:41
like non-native speaker English so that we could then analyze it, right.

10:46
And this is something that the ChatGPT can do is like generating data sets and

10:50
generating like training materials for various purposes.

10:54
And so if you’re thinking about like for you as an instructor,

10:57
if you have a task in in class that you want students to do,

11:00
getting ChatGPT to generate that, generate those materials,

11:03
then that students can analyse.

11:04
And so that was the prompt that we asked him to do was I’m sorry,

11:08
go back when I’m sorry.

11:10
You can see there that we’re using that persona pattern act as a non-native

11:13
speaker, write an e-mail.

11:14
But then we we included this include linguistic features,

11:17
problems and errors that a low intermediate level non-native speaker

11:21
would make.

11:22
So again, trying to just within the persona pattern

11:25
specify that it turns out it’s actually not very good at certain things and and

11:29
making, doing making errors in English is is one

11:31
of them.

11:32
So we weren’t, you know, yeah, we played around with it trying to get it

11:36
to to intentionally make mistakes.

11:38
And at one point it told us like, hey, I’m a I’m a I’m a I’m an AI.

11:42
I’m not very, very good at making mistakes.

11:44
I’m programmed to use standardized language or something.

11:47
And we, we’ve done various things with trying to

11:49
figure out like how do you, how does how well would it represent like,

11:52
for example, non standard varieties of English?

11:54
And so again, you can play with it and get it to kind

11:56
of push the, you know, see how far you can push it some

11:59
boundaries.

12:00
But what was interesting here, even though we thought like, well,

12:02
the errors that it represented were kind of weird.

12:05
But when we actually asked it to do then do a persona pattern as an expert persona.

12:10
And that’s another thing that it does quite well is, you know,

12:13
plays the role of of an expert.

12:16
So we said we’ll flip it around and and you be the ESL teacher and evaluate the

12:21
output that you just created as the ESL student.

12:25
It came up with a pretty good list of criteria then that it would use to

12:30
evaluate that.

12:31
And of course, those criteria then you could very easily

12:34
repackage into a rubric.

12:35
And so if you go to the next slide, Ivan, and so, you know, as, as ESL teachers,

12:40
we’re like looking at what ChatGPT and then we asked it specifically, well, what,

12:44
you know, what were the criteria?

12:46
And, you know, it gave us a, a very good list of the,

12:51
the criteria that it was using.

12:54
And again, we can just basically take those and

12:56
repackage it as a, as a rubric to, to, to use for evaluation, right?

13:00
So, so that’s one thing it, it does quite well.

13:05
You can also kind of again, you use the tweak the persona pattern a

13:09
little bit in that like flipping it around rather than instructing ChatGPT to

13:13
take on the persona of you know, you be the ESL teacher,

13:17
you be the the student.

13:18
You can say, well, imagine that I’m X persona now explain

13:22
this to me, right?

13:23
And so, for example, in linguistics, we have a,

13:26
a concept called translanguaging.

13:29
So I asked it to explain translanguaging, but explain it to a linguist and it

13:33
understands that that means explain it in like highly specialist terms, right,

13:37
using all the, the fancy jargon and everything.

13:40
And so it, you know it, it does that quite well.

13:43
But then again, you can, it’s interesting to see how you can tweak

13:46
this, explain it to a linguist versus explain

13:48
it to a layperson.

13:49
Right.

13:50
Again, comes up with a completely different but

13:53
but really good explanation of, of of this, you know,

13:56
pretty complicated theoretical concept in pretty plain everyday language.

14:01
And you can even get kind of creative with it if you say, well,

14:04
explain it not just to a layperson, but explain translanguaging,

14:06
assume I am a third grader who loves Pokémon.

14:10
And it’s like, look what it did.

14:11
It’s like translanguaging is like having special power that lets you see different

14:15
languages together, just like different Pokémon powers,

14:17
you know how Pokémon have different moves that they can use in battles.

14:20
And so like I asked my kids who’s who are Pokémon experts, right?

14:24
I was like, hey, look at it.

14:25
They’re like, yeah, that, that sounds, that sounds pretty good.

14:27
I understand translanguaging.

14:29
So it’s, it’s like really good at, you know,

14:32
getting these sort of creative mash up things that it actually, yeah,

14:36
quite smart and how it explains this linguistics concept through the sort of

14:41
the world of Pokémon and can create those sorts of mash ups, right.

14:46
So again, sometimes it’s just that it’s just the

14:48
limitation of our own creativity and thinking about how can we kind of use

14:52
these prompt engineering patterns to come up with, with different kinds of things,

14:55
right.

14:57
And so you can, you can say perform, you know, perform task X,

15:02
assume I am Persona Y.

15:04
And we, you know, did this in, in, in different, in different ways.

15:10
Ivan, I’m going to take over the screen share

15:13
here for a second so I can jump to my to my ChatGPT here.

15:18
You can see, for example, that like if I’m working with my students

15:21
and we’re, you know, working in a linguistics class and we’re

15:24
trying to understand, you know, some of the debates and linguistics

15:27
around Noam Chomsky’s views versus, you know, critiques of Chomsky, right.

15:31
And so I give it this prompt here.

15:32
Please imagine that you are Noam Chomsky.

15:34
I will ask you questions one at a time about your views on linguistics and

15:38
language.

15:39
And it says, OK, go for it.

15:40
And I said, briefly describe the difference between

15:42
competence, performance.

15:43
It gives me a nice, you know, explanation of that.

15:46
But now I start to get into like, well, you know,

15:49
are you you’ve been criticized here by these other linguists.

15:53
I said, are you prepared to admit that you were

15:55
wrong?

15:55
And he’s like, no, I wasn’t wrong.

15:57
And he and he turns around to defense as Noam Chomsky, right, defends his own,

16:02
his own of ideas here, right.

16:04
And so I’m kind of getting drawing it into this sort of debate and essentially

16:08
having sort of a a linguistics debate with Noam Chomsky or with ChatGPT and the

16:11
persona of Noam Chomsky.

16:13
Similarly, you know, I have a teacher who was a high school

16:16
social studies teacher and they’re studying about the Cuban missile crisis.

16:20
So they’re the students are able to interview John F Kennedy and Khrushchev

16:24
and ask them about, you know, what’s going on.

16:27
But so they could also ask, they asked it like, you know, “Hey, JFK,

16:30
it must be very stressful for you.

16:32
Are you sleeping OK?”

16:34
You know, you’d ask it like whatever you want,

16:36
right?

16:36
And so and allowing the students to interview these historical figures.

16:40
You want your students to interview Genghis Khan.

16:42
Sure, the ChatGPT will will do a pretty

16:45
passable job of of playing the role, you know, role-playing Genghis Khan,

16:49
right.

16:50
And so, you know, so you can see here kind of like the

16:53
potential of, you know, I lectured and explained like, oh,

16:56
there are debates and linguistics and this and that.

16:58
But like actually having the students do those debates using ChatGPT makes it just

17:03
an entirely different interactive experience on on a different level, right.

17:09
Maybe we can jump back to the to the PowerPoint there.

17:13
[Dr. Stefano] I think to the PowerPoint, yes.

17:14
[Dr. Sayer] Yeah.

17:15
So you can see kind of just by using some of these fairly simple prompt engineering

17:21
patterns, how again, you can generate some pretty amazing

17:25
kinds of activities and materials for for students.

17:29
Go, go to the next one there.

17:31
And so you can again, different versions of this.

17:36
So there’s something called what’s called the flipped interaction pattern.

17:39
But again, this is really just the persona pattern,

17:43
but give, giving it, you know, again, asking the, the,

17:47
the the ChatGPT to give you sort of directions.

17:50
And so for example, I get, you know, I consult with districts and a lot of

17:55
districts in Ohio that have a small number of English as a second language

17:59
students, they don’t have a licensed ESL person

18:01
there.

18:02
And so, you know, they’ll get a student.

18:04
Sorry, go back to the previous example, Ivan.

18:06
They’ll get a student and I’ll get an e-mail saying like, “Hey,

18:09
we’ve gotten this new,

18:10
I got an e-mail the other day.

18:10
We got this new student from Guatemala who seems like they speak some indigenous

18:14
language.

18:15
We don’t know they I don’t care.

18:16
Maybe they know some Spanish.

18:17
But like, what do we do with the student?

18:19
Right?

18:19
We want to help them.”

18:20
But.

18:21
And so you could actually ask ChatGPT again to play the role of the expert and

18:27
to ask questions until you get that kind of.

18:31
Response that, you know, until you’re able to essentially come up

18:35
with a a plan for for the student, right.

18:37
And so again, this can be used in in in lots of

18:40
different ways where you’re asking it to to be the expert to but to ask you

18:45
questions.

18:46
So, you know, for example, I took over the screen share again.

18:50
I mean, for example, you know me, you know that I’m like obsessed with

18:57
playing pickleball, right?

18:59
And so this summer I messed up my shoulder playing too much pickleball.

19:03
And so, you know, the doc said, Oh, you got frozen shoulder.

19:06
And so, you know, to go to the PT or I could, you know,

19:09
I’d go to the physical there or I could just ask ChatGPT to do it for me, right?

19:13
And so I said, ask me questions.

19:14
What?

19:14
And you’ll notice there’s little tricks here like the one at a time.

19:17
You need to kind of spell that out for it.

19:19
Otherwise it’ll start spitting out, you know, a million questions at you,

19:22
right?

19:23
But that’s basically what it did.

19:25
It’ll ask you questions and then it’ll ask you enough questions until it decides

19:29
it has enough information.

19:30
And then it gave me a a pretty good.

19:31
I actually had gone to the physical therapist.

19:33
I was able to compare what, what he gave me versus what ChatGPT came

19:37
up with.

19:37
And again, it was a pretty, you know, he ChatGPT obviously couldn’t tell,

19:41
tell me like the the PT guy could if I was doing the stretches in exactly the

19:45
right way.

19:45
But the plan itself was, was pretty comparable.

19:48
So sorry, back to that.

19:51
Sorry, we’re jumping you guys back and forth

19:53
here with that PowerPoint a lot, but can we go back to the PowerPoint?

19:56
There you go.

19:56
Thanks, Ivan.

19:57
All right, cool.

19:58
So let’s go to the next one then.

20:00
I think.

20:02
Yeah, so I think we’re we’re we’re again, those were, you know,

20:05
some of the the the most useful kind of basic prompt engineering patterns.

20:10
You can see kind of how you can combine them in different ways to come up with

20:13
different kinds of things.

20:15
And as I said, there’s there’s a lot more that that that

20:18
gets into, you know, more advanced kinds of of patterns if you

20:21
want to go even further with it.

20:25
[Dr. Stefano] So one of the questions that we

20:27
get is can you detect AI-generated text?

20:29
So this is actually a pretty interesting thing that we also experimented with

20:35
during the course.

20:38
So there you probably have heard of GPT zero and copy leaks and things like that.

20:45
The interesting thing about it is that sometimes it’s a hit or miss.

20:51
So I did generate something through ChatGPT and put it in one detector and it

20:56
says it is AI-generated and then put it in another one,

20:59
it says it’s human-generated.

21:01
So it’s really unreliable.

21:03
So the reason that we want to bring it up is because you know,

21:07
as a teacher or as an instructor, you don’t want to be in a situations

21:11
where you use this tool and then you think it’s reliable and then you’re

21:15
actually wrong.

21:16
So, so I would be really careful with that.

21:21
So this is actually, Peter, this is the next part for you, I think.

21:26
[Dr. Sayer] Well, no, it was just, I mean, again, there,

21:29
there’s been some fairly high profile examples of students being accused of

21:33
plagiarism based on those kinds of checkers that turned out to be wrong or

21:38
at least, yeah, not reliable.

21:40
I also like, for example, in that example I just gave you where my

21:43
son was doing the book report, the original version of the book report,

21:47
it said that was generated by ChatGPT, which was correct.

21:51
But as soon as he used the persona pattern and said,

21:53
write it as a 7th grader, it thought that that was human.

21:56
So if you’re using that persona pattern at all,

22:00
it’s pretty easy to to kind of trick or confuse those,

22:04
those those AI detector tools.

22:06
So something to to keep in mind that really our conclusion after we played

22:10
around with it.

22:11
Next slide, Ivan, was that you really can’t know with any.

22:17
Yeah, that was an example of one where where

22:19
again it came, it was pretty unreliable.

22:21
So yeah, unfortunately that this is kind of the

22:23
conclusion that there’s currently no way to know with any certainty what the text

22:27
has been generated by your student versus you again,

22:30
because it’s working differently than a plagiarism checker can just crawl the

22:34
Internet and figure out, OK, this is a match with this, right?

22:37
But again, the way that ChatGPT plagiarizes is not

22:41
the same way.

22:42
It’s not a copy paste kind of plagiarize.

22:44
It’s it’s a, you know, it’s, it really is a novelty generated thing.

22:47
So if the students are copy pasting that from ChatGPT, again,

22:50
a traditional plagiarism checker isn’t going to isn’t going to catch it, right?

22:56
[Dr. Stefano] So now we want to move on to the

22:58
what you can

22:59
what can you use AI for in your teachings?

23:01
I’m gonna unshare my screen here in a minute so that I can go to ChatGPT and

23:06
other tools.

23:08
One thing that some things that we found about ChatGPT ChatGPT are it’s pretty

23:13
good for at least this things and then a lot more other things,

23:17
but for education purposes, lesson planning, brainstorming,

23:20
generating discussion questions and translation, it’s actually really good at.

23:25
So I’m going to unshare my PowerPoint and I will go to my ChatGPT.

23:34
All right, so as you know, lesson planning can take some time,

23:39
but if you use ChatGPT and you use the correct prompt,

23:43
your engineer that your prompt correctly, it can cut down on your lesson planning

23:50
time.

23:50
So I’m going to go to my browser here.

23:57
So this is an example.

23:59
Can you create a lesson for my TESOL methods class?

24:02
I would like to discuss the commutative language teaching methods.

24:06
It gives you the title, the objective materials, lesson plans,

24:12
and you know, like it even allocates the how many

24:16
minutes you should use for each activity.

24:20
However, if you notice ChatGPT cannot really do

24:25
math.

24:26
It says here 90 minutes, but if you add this up,

24:29
they’re actually more than 90 minutes.

24:33
One thing that we found is that don’t do math or don’t do any calculations on

24:39
ChatGPT.

24:40
It it for some reason, it can do a lot, but that. So it’s very interesting.

24:45
And then as you have the Lesson plan here, you can continue by asking,

24:49
can you generate 10 questions for this lesson?

24:52
And then it gives you this as a teacher, however, you want to vet these.

24:58
You don’t want to just like take it.

25:00
You make sure that you actually read this and then tweak it,

25:05
modify it to to your needs and to your contacts.

25:10
Because I found that sometimes the questions are a little bit off or,

25:14
you know, they’re not kind of, they’re not focused on, you know,

25:18
what I want to do.

25:19
So, but it actually is a good start.

25:22
And then it does say save time.

25:25
Save you time when you are coming up with this question.

25:29
Another thing that we found ChatGPT is good for is for translation.

25:35
So if you know another language and you need to translate a document or a text to

25:43
to that language, I find that ChatGPT is better than Google

25:48
Translate.

25:50
So I speak Indonesian and English and sometimes I have to translate things from

25:55
Indonesian to English.

25:57
This is the the policy from OAA that was that Casey linked earlier in the or

26:02
dropped the link in the chat.

26:06
So this is in English.

26:07
And then I just put translated into Indonesian and I read this.

26:12
I would say this is like 90% to 98% accurate and I would probably just change

26:18
the few words.

26:19
So I’ve we’ve used this in our program, one of our programs where we have to

26:25
translate English text into Japanese and the, the, our,

26:29
our teacher who speaks both English and Japanese uses ChatGPT to do that.

26:35
She used to do manual translation that took hours and hours,

26:39
and now it’s just like 5 minutes and then she goes over it and then it’s done.

26:44
So I don’t know about other languages though,

26:47
but I know that she told me that Japanese is good and I know because I speak

26:52
Indonesian.

26:52
Indonesian is also good, and some other languages are probably

26:58
good too.

26:59
So this is a great use of ChatGPT.

27:04
The next thing that we want to show you is Perplexity.

27:08
So Peter, I’m going to unshare my screen.

27:13
[Dr. Sayer] So one thing that we found again,

27:16
as we were trying to figure out what ChatGPT does well and maybe what it

27:19
doesn’t do as well.

27:20
Ivan mentioned that calculation like basically your calculator does a better

27:25
job of calculating of like multiplying big numbers.

27:28
And because it’s based on an LLM as a large language model,

27:30
it’s not really what it was designed for.

27:33
But another thing that it that it is surprisingly bad at is source based

27:37
writing, which of course is academics is a lot of

27:40
what we do and term papers for students where you have to cite things.

27:44
You’ve heard about hallucinations.

27:45
You know, it it it either just makes makes up

27:49
citations or it’s just really not very good at using sources.

27:53
Because again, that’s not the way when it’s creating a

27:55
text.

27:56
That’s not the way.

27:56
It’s not writing like we do where we’ll go, you know,

27:59
find an idea somewhere in an article we read and then we cite,

28:02
bring that in and cite it.

28:04
It’s not not doing it like that.

28:05
And so again, it’s great at taking synthesizing the

28:09
ideas, but in give it in like giving credit and

28:12
proper citations and references it, it it just isn’t very good.

28:16
But there is a site that it actually does that a lot better.

28:19
This is called Perplexity AI.

28:21
And I’m not totally sure why, but I think it’s because Perplexity,

28:26
again, the ways that it’s generating the text is

28:29
just different than ChatGPT

28:31
is essentially what’s called a next word predictor.

28:33
So when it builds a sentence, it’s building it a word at a time versus

28:37
Perplexity, which does seem like it’s going to

28:39
specific sources.

28:40
So again, I do work around this linguistic concept

28:44
called translanguaging, a way of mixing languages.

28:48
And so I asked it to to explain Peter Sayer’s work on translanguaging.

28:54
And it did a very good job of of representing my word.

28:58
It also cited the relevant sources from me, but then it took it further and said,

29:03
explain the concept of translanguaging again, citing other people.

29:07
And what’s nice here is that you click on this and view sources.

29:10
And these are like real academic sources.

29:12
They’re not, you know, hallucinations that are so and it’s,

29:16
and ChatGPT is limited to its training data, which is,

29:20
has like a November for the open source 3. 5 version has a,

29:24
a November 2021 cut off date.

29:27
I can tell Perplexity is like really recent because it’s even giving me stuff

29:31
that I like.

29:32
I was in the Philippines last month doing a keynote there.

29:34
And it’s, it knows that and it’s already plugged

29:36
that in.

29:36
So it’s like it’s, it’s up to date on things.

29:39
And so here you can see that it, it’s even just figured out like what are

29:43
the, the relevant sort of extension topics?

29:46
And you can generate things there and they’ve got sources, again,

29:49
not limited just to what you had originally asked it about, but it can,

29:52
it can figure out where to go with it.

29:53
And so as far as like, if you’re building a lit review or again,

29:58
trying to come up with source based writing Perplexity is,

30:03
is organized differently and does a much better job than ChatGPT does.

30:09
Another thing that we did with with teachers and again, looking at, you know,

30:13
sources for, for, for for instructors, particularly the, you know,

30:17
I work in teacher Ed.

30:18
So there’s, you know, for K12 teachers and K12 teachers who are

30:22
working with ESL students.

30:24
This is another amazing, amazing tool which is called Diffit,

30:29
which essentially is using AI to generate not just it’s not just reading passages

30:34
or texts, but it’ll summarize it. Here you can see there’s key vocabulary

30:40
words, multiple choice questions, short answers,

30:43
and then it can generate other activities.

30:47
What’s also amazing then if I’m working with a student and I’m doing, you know,

30:51
high school, you know, what are the causes of the Civil War,

30:55
my student might be at a 10th grade level, you know, chronologically.

30:59
But again, if their English language proficiency is,

31:02
is much, much lower, essentially I can scaffold that by just

31:06
regenerating it at whatever their, their, their reading level for English is.

31:10
Or if I want them to be able to look at it in English and then see it in Spanish

31:14
or whatever their native language is, I can again translate that into.

31:19
There’s a pretty good extensive list of different languages,

31:22
even including Somali and some of the, the, the,

31:25
the less common languages that we have here in in Columbus area.

31:28
So a resource like that is pretty amazing for for teachers.

31:32
And I think Ivan, you use Twee right?

31:34
Is that the one you were going to share with?

31:36
[Dr. Stefano] Yeah.

31:36
So I’m going to show Twee.

31:37
Twee is another one, another tool that we’ve been using.

31:41
And this is for language teachers, but also could be used for other purposes.

31:47
I think.

31:48
So basically it is a tool where you can use YouTube videos or text from the

31:56
Internet and to create questions, to create discussion questions or quizzes

32:03
and things like that, which is a basically, you know,

32:09
a time saver.

32:10
So I want to show you just one thing here.

32:13
So for example, if you have a YouTube video,

32:16
so this is a YouTube video about communicative language teaching,

32:20
you paste that link there and then just you can choose open questions,

32:25
ABCD question or multiple choice true/false and then do the magic.

32:32
It gives you the question.

32:33
And this is based on the video that you have here.

32:37
And if you’re still and then it actually gives you the answers too.

32:41
So this would be good for creating a quiz and things like that.

32:45
And you know, you want to do more, go for it, read summaries.

32:53
You can do that and hold on one second.

32:58
I think you have to do the magic.

32:59
Click Do the magic again and it gives you the summary of the of the of the video.

33:08
So it only can do up to 5 minutes chunks.

33:12
So this this video was short.

33:14
So it’s 2 minutes and 8 seconds.

33:17
So it does the whole video.

33:19
But again, with anything that you generate with AI,

33:23
you do have to vet it.

33:25
You have to make sure that it is accurate because sometimes,

33:28
especially when you create multiple choice questions,

33:32
some of the distractor distractors are a little bit kind of like silly, you know,

33:37
like it’s something that students would not choose because it’s kind of like too

33:42
out there.

33:43
So this is a tool that we’ve used a lot in our language courses,

33:48
but I think I can see that this could be used for other purposes as well.

33:57
So those are some tools, but I also want to share some other

34:02
resources with you.

34:04
One thing is that one thing that we get asked a lot is like,

34:09
how do how do we know all these tools?

34:11
Like how do we find all these tools, right?

34:14
Because there’s actually literally like hundreds,

34:16
maybe thousands of them out there.

34:18
So I’m going to show you kind of like a search.

34:23
It’s not a search engine, but it’s like a list.

34:26
The websites here list AI tools one second.

34:31
Casey, would you mind putting the links in the

34:35
in the chat so that they can click right away?

34:38
OK, PowerPoint.

34:45
So futuretools. io is kind of like general AI tool search.

34:56
I’m gonna sorry, I’m going back and forth between my

34:59
browser and my browser.

35:01
[Dr. Sayer] Ivan, sorry,

35:01
I just realized we are over time too, and I know we want to leave them.

35:04
[Dr. Stefano] OK, OK, Sorry.

35:05
Yes.

35:07
So just this one thing then I’ll show you.

35:10
So this one is basically there’s like over 2000 tools and then you can filter

35:15
it to what you want and it can give you a list of tools that you want to use,

35:20
gaming, marketing, video editing for fun and things like

35:24
that.

35:25
And then the other ones in the PowerPoint that or in that Casey dropped in the chat,

35:31
you can go over them on your own.

35:35
There’s one particularly for educators.

35:38
OK yeah.

35:39
Like, sorry we’re running low on time, but we wanna leave time for questions.

35:45
I know that there are some questions in the chat,

35:47
but I wasn’t able to monitor all of them.

35:49
So if you wanna voice your questions right now, that would be great.

35:56
[Dr. Sayer] I wanted to start, though, with a point.

35:58
Channell says in the chat here, I’ve spent so much time with ChatGPT that

36:02
I can spot AI generated text in the wild.

36:06
I think you’re you’re right in that, you know,

36:08
as you’re reading this and you you kind of a student voice and, you know,

36:12
when something doesn’t, you know, yeah, you’re kind of your your,

36:15
your own sort of internal ChatGPT detector going off.

36:19
And so I think, you know, as we get better at sort of understanding

36:22
kind of how, you know, a ChatGPT voice versus the student’s

36:25
voice.

36:25
Yeah, you kind of, you know that.

36:26
Again, you can’t know for any certainty, but but I also think the other problem

36:32
though, is as students become more sophisticated

36:35
in using things like the persona pattern, they’re also going to know kind of how to,

36:41
you know, to, to basically, you know, get it to, to represent a different voice.

36:47
So, you know, I, I don’t, yeah, I think that’s the thing is like, again,

36:50
we’re, we’re being upfront and sort of like,

36:53
what are our expectations in terms of their use?

36:55
And again, you notice that what going back to what

36:58
Ivan said at the very beginning, a policy that if you’re allowing students

37:02
to use it, what’s the difference between using it as

37:05
a tool versus cheating and it including what we call an AI audit trail,

37:09
which again, is a very specific explanation of how you

37:12
were using it.

37:13
And you notice that in ChatGPT, there’s actually in the upper right hand

37:16
corner, a little share arrow.

37:18
They can actually share with you their a ChatGPT thread.

37:22
So you can see kind of what they did.

37:24
And so my students, I say, you know, go ahead and use it.

37:27
But again, do the include the AI include the the the

37:30
audit trail and even giving me give me the link so that I can look and see what

37:34
what it looks like.

37:36
[Dr. Stefano] So I think Casey, we’re we’re out of time.

37:45
[Casey Rinehart] Yeah, so I think we’ve hit that 45 mark.

37:48
You know, I’m happy to stay behind if anybody has

37:51
any questions or anything.

37:53
And, and I just also want to thank Ivan and

37:56
and Peter for this amazing presentation.

37:58
I learned so much today.

38:01
You know, our work group, we’re constantly sharing stuff and,

38:04
and sharing with our community, but every time we talk to somebody else,

38:08
there’s something new.

38:10
And that was absolutely true today.

38:12
So thank you for this wonderful presentation.

38:15
I will make sure to follow up with everyone and links of the resources that

38:19
Ivan put together and and shared in the that we shared in the chat.

38:23
And I’ll just mention as well the resources that I’ll follow up with will

38:28
include if you would like to join our EHE AI in Education Community that we post

38:34
some of this information in and our next session that will happen in February on

38:39
AI uses in your work.

38:41
So automating work and kind of how you use AI and and the everyday.

38:46
So that’s kind of a quick cap of of what we have coming your way.

38:51
Thank you again, Ivan and Peter, thank you.

38:54
We so much.

38:56
[Dr. Stefano] We could talk about this for

38:58
like hours as you can tell.

39:00
[Dr. Sayer] Yeah, I look forward to attending others.

39:02
Thanks for organizing.

39:04
[Dr. Stefano] Yeah, thank you so much for inviting us.

39:09
[Casey Rinehart] Thanks, everyone.

39:10
[Dr. Stefano] Bye everyone.