Interpretation of goal

Good morning everyone,
How would you interpret this performance review goal? "Focus on new classroom technologies with proven learning gain results".
Or better yet, what classroom technologies has your school implemented that actually resulted in "learning gain"? Like many of you (I assume), we struggle with providing classroom bells and whistles that aren't necessarily used. We've done faculty lunch and learns, and last year held a session where we asked faculty what they really want/need in classrooms. That did spark a couple of suggestions and we're now replacing many classroom monitors with SMART monitors, but I'm not sure having the ability to mark up documents really results in increased learning.
Thoughts anyone?
Thanks...

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

RE: Interpretation of goal

Cyndi,

Sorry for the delayed response to this question - and my apologies if you've already gotten a slew of responses and I'm just repeating what others have said. :)

I don't know how to interpret this as a performance review goal and I think it would be frustrating to see this as a review goal. So, if I got this, rather than spending a lot of effort on trying to understand the goal as written, I would go back to the author and request clarification. The goal as written confuses pedagogy and technology. If the class itself has no clearly stated learning objectives or goals, I think it would be very difficult to measure whether the provided technology supports the (unstated) goals of the class.

In that discussion, I would probably reference something like this: www.academicpartnerships.com/docs/default-document-library/newbooklet15_singleb.pdf . See the section on the Quality Matters rubric on page 6 or so. It does the right thing, I think, by situating technology within the context of a broader attempt to establish standards for teaching. Of course, it's oriented towards online or blended teaching, but I think the list of items is directly applicable to technology-assisted in-class teaching.

Just my two cents and I could be completely off-base. :)

Good luck!

Ben

Benjamin J. Chapman, J.D.
Assistant Dean for Information Technology, Emory University School of Law
Office: 404-727-6948 Cell/Text: 404-313-9544 IM: benjamin.chapman@gmail.com
Fewer emails in 2014: http://emailcharter.org

________________________________
From: teknoids-bounces@ruckus.law.cornell.edu <teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <johnson@law.unm.edu>
Sent: Monday, February 10, 2014 1:47 PM
To: 'Teknoids'
Subject: [teknoids] Interpretation of goal

Good morning everyone,
How would you interpret this performance review goal? "Focus on new classroom technologies with proven learning gain results".
Or better yet, what classroom technologies has your school implemented that actually resulted in "learning gain"? Like many of you (I assume), we struggle with providing classroom bells and whistles that aren't necessarily used. We've done faculty lunch and learns, and last year held a session where we asked faculty what they really want/need in classrooms. That did spark a couple of suggestions and we're now replacing many classroom monitors with SMART monitors, but I'm not sure having the ability to mark up documents really results in increased learning.
Thoughts anyone?
Thanks...

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

Ben Chapman
Assistant Dean, Information Technology
Emory University School of Law

RE: Interpretation of goal

Hi Ben and Cyndi and noids...

I don't necessarily think the goal confuses the two but - you would need a
"study" to determine whether technology enhanced learning. Ben is right in
that the subject would need defined learning outcomes and then - in theory
- if you taught with and without the technology you could see of there was
"gain." Of course, there is a huge problem. The teaching would have to be
identical EXCEPT for the use of technology. Otherwise you have variables
which are not controlled for and, as a result, you could not achieve
statistically significant results.

So - in other words - teach Subject X using technology; Teach subject X
WITHOUT the technology. Then compare the results. Then there is the
variable of students. The only way to overcome that variable is many many
data points. So you would have to conduct this study over years. And then
- once you had your results - the technology would be so out of date that
no one would want to use it. So you start over with new technology. :)
Of course that is an exaggeration but you get the idea.

Syd Beckman
Professor of Law
LMU-Duncan School of Law

On Wed, Feb 12, 2014 at 9:19 AM, Chapman, Ben <ben.chapman@emory.edu> wrote:

> Cyndi,
>
>
> Sorry for the delayed response to this question - and my apologies if
> you've already gotten a slew of responses and I'm just repeating what
> others have said. :)
>
>
> I don't know how to interpret this as a performance review goal and I
> think it would be frustrating to see this as a review goal. So, if I got
> this, rather than spending a lot of effort on trying to understand the goal
> as written, I would go back to the author and request clarification. The
> goal as written confuses pedagogy and technology. If the class itself has
> no clearly stated learning objectives or goals, I think it would be very
> difficult to measure whether the provided technology supports the
> (unstated) goals of the class.
>
>
> In that discussion, I would probably reference something like this:
> www.academicpartnerships.com/docs/default-document-library/newbooklet15_... .
> See the section on the Quality Matters rubric on page 6 or so. It does the
> right thing, I think, by situating technology within the context of a
> broader attempt to establish standards for teaching. Of course, it's
> oriented towards online or blended teaching, but I think the list of items
> is directly applicable to technology-assisted in-class teaching.
>
>
> Just my two cents and I could be completely off-base. :)
>
>
> Good luck!
>
>
> Ben
>
>
> Benjamin J. Chapman, J.D.
> Assistant Dean for Information Technology, Emory University School of Law
> Office: 404-727-6948 Cell/Text: 404-313-9544 IM:
> benjamin.chapman@gmail.com
> Fewer emails in 2014: http://emailcharter.org
>
>
> ------------------------------
> *From:* teknoids-bounces@ruckus.law.cornell.edu <
> teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <
> johnson@law.unm.edu>
> *Sent:* Monday, February 10, 2014 1:47 PM
> *To:* 'Teknoids'
> *Subject:* [teknoids] Interpretation of goal
>
>
> Good morning everyone,
>
> How would you interpret this performance review goal? "Focus on new
> classroom technologies with proven learning gain results".
>
> Or better yet, what classroom technologies has your school implemented
> that actually resulted in "learning gain"? Like many of you (I assume), we
> struggle with providing classroom bells and whistles that aren't
> necessarily used. We've done faculty lunch and learns, and last year held a
> session where we asked faculty what they really want/need in classrooms.
> That did spark a couple of suggestions and we're now replacing many
> classroom monitors with SMART monitors, but I'm not sure having the ability
> to mark up documents really results in increased learning.
>
> Thoughts anyone?
>
> Thanks...
>
>
>
> Cyndi Johnson
>
> Director/Assistant Dean for Information Technology
>
> UNM School of Law
>
> (505) 277-0695
>
>
>
> _______________________________________________
> You are currently subscribed to teknoids as: sydbeckman@gmail.com.
> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
> --
> See the web interface at
> http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list
> password, unsubscribe, and view your list settings.
>

RE: RE: Interpretation of goal

Thanks Syd, Ben, and others who replied off list...
Ben was on track when he said the goal was vague. I took it to mean that I was to walk down the path that Syd mentions below but that wasn't the intention. What I'm asked to research is how students can use technology to improve their learning. For example, he mentioned research done by his colleague at Pitt, Kevin Ashley, that studied how use of a system called Comrade, used to provide anonymous feedback from other students, can improve learning gains. So I believe the goal has nothing to do with faculty's use of technology (or lack thereof) but, rather, how students can help other students. I think the study was focused primarily on the legal writing class but I haven't had time to look at it very hard.
Of course, it seems to me that feedback provided by students is just that - student feedback - and would only be as good (or bad) as the other students in the group.
Cyndi

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf Of Syd Beckman
Sent: Wednesday, February 12, 2014 8:15 AM
To: Teknoids
Subject: Re: [teknoids] RE: Interpretation of goal

Hi Ben and Cyndi and noids...

I don't necessarily think the goal confuses the two but - you would need a "study" to determine whether technology enhanced learning. Ben is right in that the subject would need defined learning outcomes and then - in theory - if you taught with and without the technology you could see of there was "gain." Of course, there is a huge problem. The teaching would have to be identical EXCEPT for the use of technology. Otherwise you have variables which are not controlled for and, as a result, you could not achieve statistically significant results.

So - in other words - teach Subject X using technology; Teach subject X WITHOUT the technology. Then compare the results. Then there is the variable of students. The only way to overcome that variable is many many data points. So you would have to conduct this study over years. And then - once you had your results - the technology would be so out of date that no one would want to use it. So you start over with new technology. :) Of course that is an exaggeration but you get the idea.

Syd Beckman
Professor of Law
LMU-Duncan School of Law

On Wed, Feb 12, 2014 at 9:19 AM, Chapman, Ben <ben.chapman@emory.edu> wrote:

Cyndi,

Sorry for the delayed response to this question - and my apologies if you've already gotten a slew of responses and I'm just repeating what others have said. :)

I don't know how to interpret this as a performance review goal and I think it would be frustrating to see this as a review goal. So, if I got this, rather than spending a lot of effort on trying to understand the goal as written, I would go back to the author and request clarification. The goal as written confuses pedagogy and technology. If the class itself has no clearly stated learning objectives or goals, I think it would be very difficult to measure whether the provided technology supports the (unstated) goals of the class.

In that discussion, I would probably reference something like this: www.academicpartnerships.com/docs/default-document-library/newbooklet15_singleb.pdf . See the section on the Quality Matters rubric on page 6 or so. It does the right thing, I think, by situating technology within the context of a broader attempt to establish standards for teaching. Of course, it's oriented towards online or blended teaching, but I think the list of items is directly applicable to technology-assisted in-class teaching.

Just my two cents and I could be completely off-base. :)

Good luck!

Ben

Benjamin J. Chapman, J.D.
Assistant Dean for Information Technology, Emory University School of Law
Office: 404-727-6948 Cell/Text: 404-313-9544 IM: benjamin.chapman@gmail.com
Fewer emails in 2014: http://emailcharter.org

________________________________
From: teknoids-bounces@ruckus.law.cornell.edu <teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <johnson@law.unm.edu>
Sent: Monday, February 10, 2014 1:47 PM
To: 'Teknoids'
Subject: [teknoids] Interpretation of goal

Good morning everyone,
How would you interpret this performance review goal? "Focus on new classroom technologies with proven learning gain results".
Or better yet, what classroom technologies has your school implemented that actually resulted in "learning gain"? Like many of you (I assume), we struggle with providing classroom bells and whistles that aren't necessarily used. We've done faculty lunch and learns, and last year held a session where we asked faculty what they really want/need in classrooms. That did spark a couple of suggestions and we're now replacing many classroom monitors with SMART monitors, but I'm not sure having the ability to mark up documents really results in increased learning.
Thoughts anyone?
Thanks...

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

_______________________________________________
You are currently subscribed to teknoids as: sydbeckman@gmail.com.
To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
--
See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.

RE: Interpretation of goal

Kevin Ashley has been doing awesome work in Law and AI for years and
year. I had forgotten that he was doing something with legal writing and
peer-review Here are the papers...

Computer‐Supported Peer Review in a Law School Context
http://www.lrdc.pitt.edu/BOV/documents/Ashley_Computer-SupportedPeerRevi...

Toward AI - enhanced Computer supported Peer Review in Legal Education
http://www.lrdc.pitt.edu/BOV/documents/Ashley_TowardAI-Enhanced_040212.pdf

Reading now..

John

On Wed, Feb 12, 2014 at 9:30 AM, Johnson, Cyndi <johnson@law.unm.edu> wrote:

> Thanks Syd, Ben, and others who replied off list…
>
> Ben was on track when he said the goal was vague. I took it to mean that I
> was to walk down the path that Syd mentions below but that wasn’t the
> intention. What I’m asked to research is how students can use technology to
> improve their learning. For example, he mentioned research done by his
> colleague at Pitt, Kevin Ashley, that studied how use of a system called
> Comrade, used to provide anonymous feedback from other students, can
> improve learning gains. So I believe the goal has nothing to do with
> faculty’s use of technology (or lack thereof) but, rather, how students can
> help other students. I think the study was focused primarily on the legal
> writing class but I haven’t had time to look at it very hard.
>
> Of course, it seems to me that feedback provided by students is just that
> – student feedback - and would only be as good (or bad) as the other
> students in the group.
>
> Cyndi
>
>
>
> Cyndi Johnson
>
> Director/Assistant Dean for Information Technology
>
> UNM School of Law
>
> (505) 277-0695
>
>
>
> *From:* teknoids-bounces@ruckus.law.cornell.edu [mailto:
> teknoids-bounces@ruckus.law.cornell.edu] *On Behalf Of *Syd Beckman
> *Sent:* Wednesday, February 12, 2014 8:15 AM
> *To:* Teknoids
> *Subject:* Re: [teknoids] RE: Interpretation of goal
>
>
>
> Hi Ben and Cyndi and noids...
>
>
>
> I don't necessarily think the goal confuses the two but - you would need a
> "study" to determine whether technology enhanced learning. Ben is right in
> that the subject would need defined learning outcomes and then - in theory
> - if you taught with and without the technology you could see of there was
> "gain." Of course, there is a huge problem. The teaching would have to be
> identical EXCEPT for the use of technology. Otherwise you have variables
> which are not controlled for and, as a result, you could not achieve
> statistically significant results.
>
>
>
> So - in other words - teach Subject X using technology; Teach subject X
> WITHOUT the technology. Then compare the results. Then there is the
> variable of students. The only way to overcome that variable is many many
> data points. So you would have to conduct this study over years. And then
> - once you had your results - the technology would be so out of date that
> no one would want to use it. So you start over with new technology. :)
> Of course that is an exaggeration but you get the idea.
>
>
>
> Syd Beckman
>
> Professor of Law
>
> LMU-Duncan School of Law
>
>
>
>
>
> On Wed, Feb 12, 2014 at 9:19 AM, Chapman, Ben <ben.chapman@emory.edu>
> wrote:
>
> Cyndi,
>
>
>
> Sorry for the delayed response to this question - and my apologies if
> you've already gotten a slew of responses and I'm just repeating what
> others have said. :)
>
>
>
> I don't know how to interpret this as a performance review goal and I
> think it would be frustrating to see this as a review goal. So, if I got
> this, rather than spending a lot of effort on trying to understand the goal
> as written, I would go back to the author and request clarification. The
> goal as written confuses pedagogy and technology. If the class itself has
> no clearly stated learning objectives or goals, I think it would be very
> difficult to measure whether the provided technology supports the
> (unstated) goals of the class.
>
>
>
> In that discussion, I would probably reference something like this:
> www.academicpartnerships.com/docs/default-document-library/newbooklet15_... .
> See the section on the Quality Matters rubric on page 6 or so. It does the
> right thing, I think, by situating technology within the context of a
> broader attempt to establish standards for teaching. Of course, it's
> oriented towards online or blended teaching, but I think the list of items
> is directly applicable to technology-assisted in-class teaching.
>
>
>
> Just my two cents and I could be completely off-base. :)
>
>
>
> Good luck!
>
>
>
> Ben
>
>
>
> Benjamin J. Chapman, J.D.
> Assistant Dean for Information Technology, Emory University School of Law
> Office: 404-727-6948 Cell/Text: 404-313-9544 IM:
> benjamin.chapman@gmail.com
> Fewer emails in 2014: http://emailcharter.org
>
>
> ------------------------------
>
> *From:* teknoids-bounces@ruckus.law.cornell.edu <
> teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <
> johnson@law.unm.edu>
> *Sent:* Monday, February 10, 2014 1:47 PM
> *To:* 'Teknoids'
> *Subject:* [teknoids] Interpretation of goal
>
>
>
> Good morning everyone,
>
> How would you interpret this performance review goal? “Focus on new
> classroom technologies with proven learning gain results”.
>
> Or better yet, what classroom technologies has your school implemented
> that actually resulted in “learning gain”? Like many of you (I assume), we
> struggle with providing classroom bells and whistles that aren’t
> necessarily used. We’ve done faculty lunch and learns, and last year held a
> session where we asked faculty what they really want/need in classrooms.
> That did spark a couple of suggestions and we’re now replacing many
> classroom monitors with SMART monitors, but I’m not sure having the ability
> to mark up documents really results in increased learning.
>
> Thoughts anyone?
>
> Thanks…
>
>
>
> Cyndi Johnson
>
> Director/Assistant Dean for Information Technology
>
> UNM School of Law
>
> (505) 277-0695
>
>
>
>
> _______________________________________________
> You are currently subscribed to teknoids as: sydbeckman@gmail.com.
> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
> --
> See the web interface at
> http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list
> password, unsubscribe, and view your list settings.
>
>
>
> _______________________________________________
> You are currently subscribed to teknoids as: jmayer@cali.org.
> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
> --
> See the web interface at
> http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list
> password, unsubscribe, and view your list settings.
>

RE: Interpretation of goal

This is actually a very interesting question.

My first thought was what if we were talking about types of hammers. Would
you ask if a particular type of hammer improves construction? Technology
is almost never the point in education (perhaps I shouldn't hedge)...OK,
TECHNOLOGY IS NOT THE GOAL.

So asking if a particular technology improved learning is the wrong
question. First of all, learning is what happens inside the student's head
and that's impossible to measure at any deep fidelity. At best, we use
proxies like tests/assessments and interpersonal reactions ("does the
student look confused?") to "measure" learning.

Sometimes, technology can reduce the administrative overhead and quite a
bit. Before books, we had the oral tradition. Before screens, we had
chalkboards. Before powerpoint, we had arm waving (Ok, now we have both).

It's easier to show a picture than draw it from scratch every time you
teach the class (mostly) because technology.

It's easier to say "go away and look at this on the web" and have pretty
good confidence that 15 or 30 or 150 people can do that ... because
technology.

So those things are easier than the logistics of lumber, pulp, printing,
shipping, book stores, credit cards and backpacks.

Now, once you radically change the availability of vast amounts of
information, you change the way that teaching can happen and can only begin
to explore if more/better learning is happening, and we are back to proxies
of measurement.

I'm not helping here I think.

I would interpret this as HR wants an answer that demonstrates that you are
thinking about the core goals and mission of the university/law school and
that you are more than a brain stem punching buttons. Your desire to
understand and answer this question shows exactly that. I would turn this
into a discussion of the issues instead of trying to give a pat answer.
There is no pat answer,so give them a buffet of answers.

John

On Wed, Feb 12, 2014 at 9:30 AM, Johnson, Cyndi <johnson@law.unm.edu> wrote:

> Thanks Syd, Ben, and others who replied off list...
>
> Ben was on track when he said the goal was vague. I took it to mean that I
> was to walk down the path that Syd mentions below but that wasn't the
> intention. What I'm asked to research is how students can use technology to
> improve their learning. For example, he mentioned research done by his
> colleague at Pitt, Kevin Ashley, that studied how use of a system called
> Comrade, used to provide anonymous feedback from other students, can
> improve learning gains. So I believe the goal has nothing to do with
> faculty's use of technology (or lack thereof) but, rather, how students can
> help other students. I think the study was focused primarily on the legal
> writing class but I haven't had time to look at it very hard.
>
> Of course, it seems to me that feedback provided by students is just that
> - student feedback - and would only be as good (or bad) as the other
> students in the group.
>
> Cyndi
>
>
>
> Cyndi Johnson
>
> Director/Assistant Dean for Information Technology
>
> UNM School of Law
>
> (505) 277-0695
>
>
>
> *From:* teknoids-bounces@ruckus.law.cornell.edu [mailto:
> teknoids-bounces@ruckus.law.cornell.edu] *On Behalf Of *Syd Beckman
> *Sent:* Wednesday, February 12, 2014 8:15 AM
> *To:* Teknoids
> *Subject:* Re: [teknoids] RE: Interpretation of goal
>
>
>
> Hi Ben and Cyndi and noids...
>
>
>
> I don't necessarily think the goal confuses the two but - you would need a
> "study" to determine whether technology enhanced learning. Ben is right in
> that the subject would need defined learning outcomes and then - in theory
> - if you taught with and without the technology you could see of there was
> "gain." Of course, there is a huge problem. The teaching would have to be
> identical EXCEPT for the use of technology. Otherwise you have variables
> which are not controlled for and, as a result, you could not achieve
> statistically significant results.
>
>
>
> So - in other words - teach Subject X using technology; Teach subject X
> WITHOUT the technology. Then compare the results. Then there is the
> variable of students. The only way to overcome that variable is many many
> data points. So you would have to conduct this study over years. And then
> - once you had your results - the technology would be so out of date that
> no one would want to use it. So you start over with new technology. :)
> Of course that is an exaggeration but you get the idea.
>
>
>
> Syd Beckman
>
> Professor of Law
>
> LMU-Duncan School of Law
>
>
>
>
>
> On Wed, Feb 12, 2014 at 9:19 AM, Chapman, Ben <ben.chapman@emory.edu>
> wrote:
>
> Cyndi,
>
>
>
> Sorry for the delayed response to this question - and my apologies if
> you've already gotten a slew of responses and I'm just repeating what
> others have said. :)
>
>
>
> I don't know how to interpret this as a performance review goal and I
> think it would be frustrating to see this as a review goal. So, if I got
> this, rather than spending a lot of effort on trying to understand the goal
> as written, I would go back to the author and request clarification. The
> goal as written confuses pedagogy and technology. If the class itself has
> no clearly stated learning objectives or goals, I think it would be very
> difficult to measure whether the provided technology supports the
> (unstated) goals of the class.
>
>
>
> In that discussion, I would probably reference something like this:
> www.academicpartnerships.com/docs/default-document-library/newbooklet15_... .
> See the section on the Quality Matters rubric on page 6 or so. It does the
> right thing, I think, by situating technology within the context of a
> broader attempt to establish standards for teaching. Of course, it's
> oriented towards online or blended teaching, but I think the list of items
> is directly applicable to technology-assisted in-class teaching.
>
>
>
> Just my two cents and I could be completely off-base. :)
>
>
>
> Good luck!
>
>
>
> Ben
>
>
>
> Benjamin J. Chapman, J.D.
> Assistant Dean for Information Technology, Emory University School of Law
> Office: 404-727-6948 Cell/Text: 404-313-9544 IM:
> benjamin.chapman@gmail.com
> Fewer emails in 2014: http://emailcharter.org
>
>
> ------------------------------
>
> *From:* teknoids-bounces@ruckus.law.cornell.edu <
> teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <
> johnson@law.unm.edu>
> *Sent:* Monday, February 10, 2014 1:47 PM
> *To:* 'Teknoids'
> *Subject:* [teknoids] Interpretation of goal
>
>
>
> Good morning everyone,
>
> How would you interpret this performance review goal? "Focus on new
> classroom technologies with proven learning gain results".
>
> Or better yet, what classroom technologies has your school implemented
> that actually resulted in "learning gain"? Like many of you (I assume), we
> struggle with providing classroom bells and whistles that aren't
> necessarily used. We've done faculty lunch and learns, and last year held a
> session where we asked faculty what they really want/need in classrooms.
> That did spark a couple of suggestions and we're now replacing many
> classroom monitors with SMART monitors, but I'm not sure having the ability
> to mark up documents really results in increased learning.
>
> Thoughts anyone?
>
> Thanks...
>
>
>
> Cyndi Johnson
>
> Director/Assistant Dean for Information Technology
>
> UNM School of Law
>
> (505) 277-0695
>
>
>
>
> _______________________________________________
> You are currently subscribed to teknoids as: sydbeckman@gmail.com.
> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
> --
> See the web interface at
> http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list
> password, unsubscribe, and view your list settings.
>
>
>
> _______________________________________________
> You are currently subscribed to teknoids as: jmayer@cali.org.
> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
> --
> See the web interface at
> http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list
> password, unsubscribe, and view your list settings.
>

RE: RE: Interpretation of goal

It is an interesting topic. I think the instructional technologists on this list can provide more details than I can on how technology can enhance instruction and there's probably more research based around the impact of technology on learning outcomes than I want to dive into. Those are the proxies of measurement you reference, right?
My quick take-away is that students helping students is old news. Tutors, discussions, study groups, debates outside of class...nothing new there. What's interesting to me is the idea that technology can play a role in how students help students in ways I haven't thought of. Look at how people interact now. Sure, lots of sitting around and talking. But I'd guess - no, I know - that our students are connected to each other via Twitter, FB, Snapchat, etc etc etc. Why not use that interest in a way that can help them improve their analytical skills? Maybe not "learn more" or have a deeper understanding...that's too vague. But skills are measurable.
Cyndi

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf Of John Mayer
Sent: Wednesday, February 12, 2014 1:20 PM
To: Teknoids
Subject: Re: [teknoids] RE: Interpretation of goal

This is actually a very interesting question.

My first thought was what if we were talking about types of hammers. Would you ask if a particular type of hammer improves construction? Technology is almost never the point in education (perhaps I shouldn't hedge)...OK, TECHNOLOGY IS NOT THE GOAL.

So asking if a particular technology improved learning is the wrong question. First of all, learning is what happens inside the student's head and that's impossible to measure at any deep fidelity. At best, we use proxies like tests/assessments and interpersonal reactions ("does the student look confused?") to "measure" learning.

Sometimes, technology can reduce the administrative overhead and quite a bit. Before books, we had the oral tradition. Before screens, we had chalkboards. Before powerpoint, we had arm waving (Ok, now we have both).

It's easier to show a picture than draw it from scratch every time you teach the class (mostly) because technology.

It's easier to say "go away and look at this on the web" and have pretty good confidence that 15 or 30 or 150 people can do that ... because technology.

So those things are easier than the logistics of lumber, pulp, printing, shipping, book stores, credit cards and backpacks.

Now, once you radically change the availability of vast amounts of information, you change the way that teaching can happen and can only begin to explore if more/better learning is happening, and we are back to proxies of measurement.

I'm not helping here I think.

I would interpret this as HR wants an answer that demonstrates that you are thinking about the core goals and mission of the university/law school and that you are more than a brain stem punching buttons. Your desire to understand and answer this question shows exactly that. I would turn this into a discussion of the issues instead of trying to give a pat answer. There is no pat answer,so give them a buffet of answers.

John

On Wed, Feb 12, 2014 at 9:30 AM, Johnson, Cyndi <johnson@law.unm.edu> wrote:
Thanks Syd, Ben, and others who replied off list...
Ben was on track when he said the goal was vague. I took it to mean that I was to walk down the path that Syd mentions below but that wasn't the intention. What I'm asked to research is how students can use technology to improve their learning. For example, he mentioned research done by his colleague at Pitt, Kevin Ashley, that studied how use of a system called Comrade, used to provide anonymous feedback from other students, can improve learning gains. So I believe the goal has nothing to do with faculty's use of technology (or lack thereof) but, rather, how students can help other students. I think the study was focused primarily on the legal writing class but I haven't had time to look at it very hard.
Of course, it seems to me that feedback provided by students is just that - student feedback - and would only be as good (or bad) as the other students in the group.
Cyndi

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf Of Syd Beckman
Sent: Wednesday, February 12, 2014 8:15 AM
To: Teknoids
Subject: Re: [teknoids] RE: Interpretation of goal

Hi Ben and Cyndi and noids...

I don't necessarily think the goal confuses the two but - you would need a "study" to determine whether technology enhanced learning. Ben is right in that the subject would need defined learning outcomes and then - in theory - if you taught with and without the technology you could see of there was "gain." Of course, there is a huge problem. The teaching would have to be identical EXCEPT for the use of technology. Otherwise you have variables which are not controlled for and, as a result, you could not achieve statistically significant results.

So - in other words - teach Subject X using technology; Teach subject X WITHOUT the technology. Then compare the results. Then there is the variable of students. The only way to overcome that variable is many many data points. So you would have to conduct this study over years. And then - once you had your results - the technology would be so out of date that no one would want to use it. So you start over with new technology. :) Of course that is an exaggeration but you get the idea.

Syd Beckman
Professor of Law
LMU-Duncan School of Law

On Wed, Feb 12, 2014 at 9:19 AM, Chapman, Ben <ben.chapman@emory.edu> wrote:

Cyndi,

Sorry for the delayed response to this question - and my apologies if you've already gotten a slew of responses and I'm just repeating what others have said. :)

I don't know how to interpret this as a performance review goal and I think it would be frustrating to see this as a review goal. So, if I got this, rather than spending a lot of effort on trying to understand the goal as written, I would go back to the author and request clarification. The goal as written confuses pedagogy and technology. If the class itself has no clearly stated learning objectives or goals, I think it would be very difficult to measure whether the provided technology supports the (unstated) goals of the class.

In that discussion, I would probably reference something like this: www.academicpartnerships.com/docs/default-document-library/newbooklet15_singleb.pdf . See the section on the Quality Matters rubric on page 6 or so. It does the right thing, I think, by situating technology within the context of a broader attempt to establish standards for teaching. Of course, it's oriented towards online or blended teaching, but I think the list of items is directly applicable to technology-assisted in-class teaching.

Just my two cents and I could be completely off-base. :)

Good luck!

Ben

Benjamin J. Chapman, J.D.
Assistant Dean for Information Technology, Emory University School of Law
Office: 404-727-6948 Cell/Text: 404-313-9544 IM: benjamin.chapman@gmail.com
Fewer emails in 2014: http://emailcharter.org

________________________________
From: teknoids-bounces@ruckus.law.cornell.edu <teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <johnson@law.unm.edu>
Sent: Monday, February 10, 2014 1:47 PM
To: 'Teknoids'
Subject: [teknoids] Interpretation of goal

Good morning everyone,
How would you interpret this performance review goal? "Focus on new classroom technologies with proven learning gain results".
Or better yet, what classroom technologies has your school implemented that actually resulted in "learning gain"? Like many of you (I assume), we struggle with providing classroom bells and whistles that aren't necessarily used. We've done faculty lunch and learns, and last year held a session where we asked faculty what they really want/need in classrooms. That did spark a couple of suggestions and we're now replacing many classroom monitors with SMART monitors, but I'm not sure having the ability to mark up documents really results in increased learning.
Thoughts anyone?
Thanks...

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

_______________________________________________
You are currently subscribed to teknoids as: sydbeckman@gmail.com.
To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
--
See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.

_______________________________________________
You are currently subscribed to teknoids as: jmayer@cali.org.
To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
--
See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.

--
----------
John Mayer
Executive Director
Center for Computer-Assisted Legal Instruction/CALI
565 West Adams
Chicago, IL 60661
312-906-5307
312-906-5280 - fax
jmayer@cali.org
http://www.cali.org
twitter.com/johnpmayer
----------

Interpretation of goal

Below are a couple of paragraphs from a lit review I recently completed in preparation for some research I’m hoping to do this year on the effectiveness of using a flipped pedagogy in information literacy instruction. In short, my take on the literature is that new technologies don't make direct impact on student learning outcomes, however new technologies can enable new pedagogies that can have significant impacts on learning outcomes. I was tempted to title one of the sections in document, “Education Technology - It’s the Pedagogy Stupid!" http://goo.gl/Iy9NMc

A Historical Perspective
When Clark (1994) defended his argument that the delivery media for instruction usually does not make a difference in, the technological landscape was much different. VHS video tapes were the primary means of watching on demand educational videos. Microsoft Windows 3.11 was the dominant desktop operating system, and dial-up modems using phone lines achieving speeds of 0.028 Mps were state of the art (compared to 2-20 Mps in 2013). At the same time, educational content was just starting to be distributed through relatively expensive multi-media CD-ROM applications for Windows and Macintosh. I suspect that in his era, Clark’s assessment that technology does not improve instruction was close to 100% true. On the other hand, with 20+ years of maturation and significant improvements in bandwidth, hardware speed, and authoring tool usability improvements, technology is now in a position to make a large positive impact in the delivery of instruction by enabling new pedagogical approaches to instruction, like Flipped Classrooms and Problem Based Learning with virtual simulations and collaboration (Becker, 2010).
Pedagogy, Not Technology Key. While the passage of time has been kind to the pro-technology arguments of Kosma (1994), it is important to remember Clark (1994) was correct in arguing that no matter what new technology we use, if we do not also change pedagogy, the educational outcomes will stay the same (Oblinger & Hawkins, 2006). An example of this is a study that was conducted at the University of North Texas where a comparison of student retention of Information Literacy skills was measured between sections instructed in a traditional Face-to-Face class, a blended class, and an online class. In each of the three classes, the instructional materials and pedagogy were kept as uniform as possible. Not surprisingly, the researchers found that there was no significant difference in IL skills retention between the three different lecture delivery methods (Anderson & May, 2010).
In the large meta-study conducted by the US Department of education by Means et al. (2010), blended classes were found to have a statistically significantly higher summative assessment scores than FtF classes (2010). Given the lack of information about pedagogies used in the hundreds of studies they analysed, they stated that “the observed advantage for blended learning conditions is not necessarily rooted in the media user per se and may reflect differences in content, pedagogy and learning time” (p. xv). Happily for researchers, most post-2010 studies disclose more details about both media and pedagogies employed.
Typically, early in the adoption of new technologies, we do not take advantage of all the new capabilities available to us and tend to mimic activities that we are familiar with (Oblinger & Hawkins, 2006). A recent example of this is educators creating instructional materials for Massively Online Open Courses (MOOC) by videotaping complete FtF lectures and putting them online rather than taking advantage of the flexibility of the digital medium and dividing it into a number of shorter segments, adding extra audio and visual elements to enrich the instruction.

http://goo.gl/Iy9NMc
-----
Becker, K. (n.d.). The Clark-Kozma debate in the 21st century. Retrieved November 15, 2013, from https://www.academia.edu/462857/The_Clark-Kozma_Debate_in_the_21st_Century

Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29. doi:10.1007/BF02299088

Means, B., Yoyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. US Department of Education. Retrieved from http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalrepo...

Oblinger, D., & Hawkins, B. (2006). IT myths: The myth about no significant difference. Educause Review, 41(6), 14–15.

---
Rich McCue, BComm
University of Victoria Libraries
http://library.uvic.ca
http://richmccue.com

My PGP Public Key: http://msys.ca/PGPKeyRichMcCueUvicPublic.asc

On Feb 12, 2014, at 12:35 PM, Johnson, Cyndi <johnson@law.unm.edu> wrote:

> It is an interesting topic. I think the instructional technologists on this list can provide more details than I can on how technology can enhance instruction and there’s probably more research based around the impact of technology on learning outcomes than I want to dive into. Those are the proxies of measurement you reference, right?
> My quick take-away is that students helping students is old news. Tutors, discussions, study groups, debates outside of class…nothing new there. What’s interesting to me is the idea that technology can play a role in how students help students in ways I haven’t thought of. Look at how people interact now. Sure, lots of sitting around and talking. But I’d guess – no, I know – that our students are connected to each other via Twitter, FB, Snapchat, etc etc etc. Why not use that interest in a way that can help them improve their analytical skills? Maybe not “learn more” or have a deeper understanding…that’s too vague. But skills are measurable.
> Cyndi
>
> Cyndi Johnson
> Director/Assistant Dean for Information Technology
> UNM School of Law
> (505) 277-0695
>
> From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfJohn Mayer
> Sent: Wednesday, February 12, 2014 1:20 PM
> To: Teknoids
> Subject: Re: [teknoids] RE: Interpretation of goal
>
> This is actually a very interesting question.
>
> My first thought was what if we were talking about types of hammers. Would you ask if a particular type of hammer improves construction? Technology is almost never the point in education (perhaps I shouldn't hedge)...OK, TECHNOLOGY IS NOT THE GOAL.
>
> So asking if a particular technology improved learning is the wrong question. First of all, learning is what happens inside the student's head and that's impossible to measure at any deep fidelity. At best, we use proxies like tests/assessments and interpersonal reactions ("does the student look confused?") to "measure" learning.
>
> Sometimes, technology can reduce the administrative overhead and quite a bit. Before books, we had the oral tradition. Before screens, we had chalkboards. Before powerpoint, we had arm waving (Ok, now we have both).
>
> It's easier to show a picture than draw it from scratch every time you teach the class (mostly) because technology.
>
> It's easier to say "go away and look at this on the web" and have pretty good confidence that 15 or 30 or 150 people can do that ... because technology.
>
> So those things are easier than the logistics of lumber, pulp, printing, shipping, book stores, credit cards and backpacks.
>
> Now, once you radically change the availability of vast amounts of information, you change the way that teaching can happen and can only begin to explore if more/better learning is happening, and we are back to proxies of measurement.
>
> I'm not helping here I think.
>
> I would interpret this as HR wants an answer that demonstrates that you are thinking about the core goals and mission of the university/law school and that you are more than a brain stem punching buttons. Your desire to understand and answer this question shows exactly that. I would turn this into a discussion of the issues instead of trying to give a pat answer. There is no pat answer,so give them a buffet of answers.
>
> John
>
>
>
>
>
>
> On Wed, Feb 12, 2014 at 9:30 AM, Johnson, Cyndi <johnson@law.unm.edu> wrote:
> Thanks Syd, Ben, and others who replied off list…
> Ben was on track when he said the goal was vague. I took it to mean that I was to walk down the path that Syd mentions below but that wasn’t the intention. What I’m asked to research is how students can use technology to improve their learning. For example, he mentioned research done by his colleague at Pitt, Kevin Ashley, that studied how use of a system called Comrade, used to provide anonymous feedback from other students, can improve learning gains. So I believe the goal has nothing to do with faculty’s use of technology (or lack thereof) but, rather, how students can help other students. I think the study was focused primarily on the legal writing class but I haven’t had time to look at it very hard.
> Of course, it seems to me that feedback provided by students is just that – student feedback - and would only be as good (or bad) as the other students in the group.
> Cyndi
>
> Cyndi Johnson
> Director/Assistant Dean for Information Technology
> UNM School of Law
> (505) 277-0695
>
> From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfSyd Beckman
> Sent: Wednesday, February 12, 2014 8:15 AM
> To: Teknoids
> Subject: Re: [teknoids] RE: Interpretation of goal
>
> Hi Ben and Cyndi and noids...
>
> I don't necessarily think the goal confuses the two but - you would need a "study" to determine whether technology enhanced learning. Ben is right in that the subject would need defined learning outcomes and then - in theory - if you taught with and without the technology you could see of there was "gain." Of course, there is a huge problem. The teaching would have to be identical EXCEPT for the use of technology. Otherwise you have variables which are not controlled for and, as a result, you could not achieve statistically significant results.
>
> So - in other words - teach Subject X using technology; Teach subject X WITHOUT the technology. Then compare the results. Then there is the variable of students. The only way to overcome that variable is many many data points. So you would have to conduct this study over years. And then - once you had your results - the technology would be so out of date that no one would want to use it. So you start over with new technology. :) Of course that is an exaggeration but you get the idea.
>
> Syd Beckman
> Professor of Law
> LMU-Duncan School of Law
>
>
>
> On Wed, Feb 12, 2014 at 9:19 AM, Chapman, Ben <ben.chapman@emory.edu> wrote:
> Cyndi,
>
>
>
> Sorry for the delayed response to this question - and my apologies if you've already gotten a slew of responses and I'm just repeating what others have said. :)
>
>
>
> I don't know how to interpret this as a performance review goal and I think it would be frustrating to see this as a review goal. So, if I got this, rather than spending a lot of effort on trying to understand the goal as written, I would go back to the author and request clarification. The goal as written confuses pedagogy and technology. If the class itself has no clearly stated learning objectives or goals, I think it would be very difficult to measure whether the provided technology supports the (unstated) goals of the class.
>
>
>
> In that discussion, I would probably reference something like this: www.academicpartnerships.com/docs/default-document-library/newbooklet15_... . See the section on the Quality Matters rubric on page 6 or so. It does the right thing, I think, by situating technology within the context of a broader attempt to establish standards for teaching. Of course, it's oriented towards online or blended teaching, but I think the list of items is directly applicable to technology-assisted in-class teaching.
>
>
>
> Just my two cents and I could be completely off-base. :)
>
>
>
> Good luck!
>
>
>
> Ben
>
>
>
> Benjamin J. Chapman, J.D.
> Assistant Dean for Information Technology, Emory University School of Law
> Office: 404-727-6948 Cell/Text: 404-313-9544 IM: benjamin.chapman@gmail.com
> Fewer emails in 2014: http://emailcharter.org
>
>
> From: teknoids-bounces@ruckus.law.cornell.edu <teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <johnson@law.unm.edu>
> Sent: Monday, February 10, 2014 1:47 PM
> To: 'Teknoids'
> Subject: [teknoids] Interpretation of goal
>
> Good morning everyone,
> How would you interpret this performance review goal? “Focus on new classroom technologies with proven learning gain results”.
> Or better yet, what classroom technologies has your school implemented that actually resulted in “learning gain”? Like many of you (I assume), we struggle with providing classroom bells and whistles that aren’t necessarily used. We’ve done faculty lunch and learns, and last year held a session where we asked faculty what they really want/need in classrooms. That did spark a couple of suggestions and we’re now replacing many classroom monitors with SMART monitors, but I’m not sure having the ability to mark up documents really results in increased learning.
> Thoughts anyone?
> Thanks…
>
> Cyndi Johnson
> Director/Assistant Dean for Information Technology
> UNM School of Law
> (505) 277-0695
>
>
> _______________________________________________
> You are currently subscribed to teknoids as: sydbeckman@gmail.com.
> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
> --
> See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.
>
>
> _______________________________________________
> You are currently subscribed to teknoids as: jmayer@cali.org.
> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
> --
> See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.
>
>
>

Interpretation of goal

Sorry everyone… I somehow missed the paragraph that sets up everything below including a wonderful technology as a truck metaphor:

When trying to determine if a technology contributes to the effectiveness of instruction, the issue of “no difference expected” made famous by the Clark, Kozma (1994) debate needs to be addressed. Clark (1994) argued that “media are mere vehicles that deliver instruction” and that “student achievement [is not influenced by a new delivery technology like video] any more than the truck that delivers our groceries causes changes in our nutrition” (p. 22). Kozma (1994) countered that while Clark’s argument is often correct, “if media are going to influence learning, media must be designed to give us powerful new methods, and our methods must take appropriate advantage of a medium’s capabilities” (p.16).

---
Rich McCue, BComm
University of Victoria Libraries
http://library.uvic.ca
http://richmccue.com

My PGP Public Key: http://msys.ca/PGPKeyRichMcCueUvicPublic.asc

On Feb 12, 2014, at 12:57 PM, Rich McCue <rmccue@uvic.ca> wrote:

> Below are a couple of paragraphs from a lit review I recently completed in preparation for some research I’m hoping to do this year on the effectiveness of using a flipped pedagogy in information literacy instruction. In short, my take on the literature is that new technologies don't make direct impact on student learning outcomes, however new technologies can enable new pedagogies that can have significant impacts on learning outcomes. I was tempted to title one of the sections in document, “Education Technology - It’s the Pedagogy Stupid!" http://goo.gl/Iy9NMc
>
> A Historical Perspective
> When Clark (1994) defended his argument that the delivery media for instruction usually does not make a difference in, the technological landscape was much different. VHS video tapes were the primary means of watching on demand educational videos. Microsoft Windows 3.11 was the dominant desktop operating system, and dial-up modems using phone lines achieving speeds of 0.028 Mps were state of the art (compared to 2-20 Mps in 2013). At the same time, educational content was just starting to be distributed through relatively expensive multi-media CD-ROM applications for Windows and Macintosh. I suspect that in his era, Clark’s assessment that technology does not improve instruction was close to 100% true. On the other hand, with 20+ years of maturation and significant improvements in bandwidth, hardware speed, and authoring tool usability improvements, technology is now in a position to make a large positive impact in the delivery of instruction by enabling new pedagogical approaches to instruction, like Flipped Classrooms and Problem Based Learning with virtual simulations and collaboration (Becker, 2010).
> Pedagogy, Not Technology Key. While the passage of time has been kind to the pro-technology arguments of Kosma (1994), it is important to remember Clark (1994) was correct in arguing that no matter what new technology we use, if we do not also change pedagogy, the educational outcomes will stay the same (Oblinger & Hawkins, 2006). An example of this is a study that was conducted at the University of North Texas where a comparison of student retention of Information Literacy skills was measured between sections instructed in a traditional Face-to-Face class, a blended class, and an online class. In each of the three classes, the instructional materials and pedagogy were kept as uniform as possible. Not surprisingly, the researchers found that there was no significant difference in IL skills retention between the three different lecture delivery methods (Anderson & May, 2010).
> In the large meta-study conducted by the US Department of education by Means et al. (2010), blended classes were found to have a statistically significantly higher summative assessment scores than FtF classes (2010). Given the lack of information about pedagogies used in the hundreds of studies they analysed, they stated that “the observed advantage for blended learning conditions is not necessarily rooted in the media user per se and may reflect differences in content, pedagogy and learning time” (p. xv). Happily for researchers, most post-2010 studies disclose more details about both media and pedagogies employed.
> Typically, early in the adoption of new technologies, we do not take advantage of all the new capabilities available to us and tend to mimic activities that we are familiar with (Oblinger & Hawkins, 2006). A recent example of this is educators creating instructional materials for Massively Online Open Courses (MOOC) by videotaping complete FtF lectures and putting them online rather than taking advantage of the flexibility of the digital medium and dividing it into a number of shorter segments, adding extra audio and visual elements to enrich the instruction.
>
> http://goo.gl/Iy9NMc
> -----
> Becker, K. (n.d.). The Clark-Kozma debate in the 21st century. Retrieved November 15, 2013, from https://www.academia.edu/462857/The_Clark-Kozma_Debate_in_the_21st_Century
>
> Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29. doi:10.1007/BF02299088
>
> Means, B., Yoyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. US Department of Education. Retrieved from http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalrepo...
>
> Oblinger, D., & Hawkins, B. (2006). IT myths: The myth about no significant difference. Educause Review, 41(6), 14–15.
>
>
>
>
>
> ---
> Rich McCue, BComm
> University of Victoria Libraries
> http://library.uvic.ca
> http://richmccue.com
>
> My PGP Public Key: http://msys.ca/PGPKeyRichMcCueUvicPublic.asc
>
>
> On Feb 12, 2014, at 12:35 PM, Johnson, Cyndi <johnson@law.unm.edu> wrote:
>
>> It is an interesting topic. I think the instructional technologists on this list can provide more details than I can on how technology can enhance instruction and there’s probably more research based around the impact of technology on learning outcomes than I want to dive into. Those are the proxies of measurement you reference, right?
>> My quick take-away is that students helping students is old news. Tutors, discussions, study groups, debates outside of class…nothing new there. What’s interesting to me is the idea that technology can play a role in how students help students in ways I haven’t thought of. Look at how people interact now. Sure, lots of sitting around and talking. But I’d guess – no, I know – that our students are connected to each other via Twitter, FB, Snapchat, etc etc etc. Why not use that interest in a way that can help them improve their analytical skills? Maybe not “learn more” or have a deeper understanding…that’s too vague. But skills are measurable.
>> Cyndi
>>
>> Cyndi Johnson
>> Director/Assistant Dean for Information Technology
>> UNM School of Law
>> (505) 277-0695
>>
>> From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfJohn Mayer
>> Sent: Wednesday, February 12, 2014 1:20 PM
>> To: Teknoids
>> Subject: Re: [teknoids] RE: Interpretation of goal
>>
>> This is actually a very interesting question.
>>
>> My first thought was what if we were talking about types of hammers. Would you ask if a particular type of hammer improves construction? Technology is almost never the point in education (perhaps I shouldn't hedge)...OK, TECHNOLOGY IS NOT THE GOAL.
>>
>> So asking if a particular technology improved learning is the wrong question. First of all, learning is what happens inside the student's head and that's impossible to measure at any deep fidelity. At best, we use proxies like tests/assessments and interpersonal reactions ("does the student look confused?") to "measure" learning.
>>
>> Sometimes, technology can reduce the administrative overhead and quite a bit. Before books, we had the oral tradition. Before screens, we had chalkboards. Before powerpoint, we had arm waving (Ok, now we have both).
>>
>> It's easier to show a picture than draw it from scratch every time you teach the class (mostly) because technology.
>>
>> It's easier to say "go away and look at this on the web" and have pretty good confidence that 15 or 30 or 150 people can do that ... because technology.
>>
>> So those things are easier than the logistics of lumber, pulp, printing, shipping, book stores, credit cards and backpacks.
>>
>> Now, once you radically change the availability of vast amounts of information, you change the way that teaching can happen and can only begin to explore if more/better learning is happening, and we are back to proxies of measurement.
>>
>> I'm not helping here I think.
>>
>> I would interpret this as HR wants an answer that demonstrates that you are thinking about the core goals and mission of the university/law school and that you are more than a brain stem punching buttons. Your desire to understand and answer this question shows exactly that. I would turn this into a discussion of the issues instead of trying to give a pat answer. There is no pat answer,so give them a buffet of answers.
>>
>> John
>>
>>
>>
>>
>>
>>
>> On Wed, Feb 12, 2014 at 9:30 AM, Johnson, Cyndi <johnson@law.unm.edu> wrote:
>> Thanks Syd, Ben, and others who replied off list…
>> Ben was on track when he said the goal was vague. I took it to mean that I was to walk down the path that Syd mentions below but that wasn’t the intention. What I’m asked to research is how students can use technology to improve their learning. For example, he mentioned research done by his colleague at Pitt, Kevin Ashley, that studied how use of a system called Comrade, used to provide anonymous feedback from other students, can improve learning gains. So I believe the goal has nothing to do with faculty’s use of technology (or lack thereof) but, rather, how students can help other students. I think the study was focused primarily on the legal writing class but I haven’t had time to look at it very hard.
>> Of course, it seems to me that feedback provided by students is just that – student feedback - and would only be as good (or bad) as the other students in the group.
>> Cyndi
>>
>> Cyndi Johnson
>> Director/Assistant Dean for Information Technology
>> UNM School of Law
>> (505) 277-0695
>>
>> From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfSyd Beckman
>> Sent: Wednesday, February 12, 2014 8:15 AM
>> To: Teknoids
>> Subject: Re: [teknoids] RE: Interpretation of goal
>>
>> Hi Ben and Cyndi and noids...
>>
>> I don't necessarily think the goal confuses the two but - you would need a "study" to determine whether technology enhanced learning. Ben is right in that the subject would need defined learning outcomes and then - in theory - if you taught with and without the technology you could see of there was "gain." Of course, there is a huge problem. The teaching would have to be identical EXCEPT for the use of technology. Otherwise you have variables which are not controlled for and, as a result, you could not achieve statistically significant results.
>>
>> So - in other words - teach Subject X using technology; Teach subject X WITHOUT the technology. Then compare the results. Then there is the variable of students. The only way to overcome that variable is many many data points. So you would have to conduct this study over years. And then - once you had your results - the technology would be so out of date that no one would want to use it. So you start over with new technology. :) Of course that is an exaggeration but you get the idea.
>>
>> Syd Beckman
>> Professor of Law
>> LMU-Duncan School of Law
>>
>>
>>
>> On Wed, Feb 12, 2014 at 9:19 AM, Chapman, Ben <ben.chapman@emory.edu> wrote:
>> Cyndi,
>>
>>
>>
>> Sorry for the delayed response to this question - and my apologies if you've already gotten a slew of responses and I'm just repeating what others have said. :)
>>
>>
>>
>> I don't know how to interpret this as a performance review goal and I think it would be frustrating to see this as a review goal. So, if I got this, rather than spending a lot of effort on trying to understand the goal as written, I would go back to the author and request clarification. The goal as written confuses pedagogy and technology. If the class itself has no clearly stated learning objectives or goals, I think it would be very difficult to measure whether the provided technology supports the (unstated) goals of the class.
>>
>>
>>
>> In that discussion, I would probably reference something like this: www.academicpartnerships.com/docs/default-document-library/newbooklet15_... . See the section on the Quality Matters rubric on page 6 or so. It does the right thing, I think, by situating technology within the context of a broader attempt to establish standards for teaching. Of course, it's oriented towards online or blended teaching, but I think the list of items is directly applicable to technology-assisted in-class teaching.
>>
>>
>>
>> Just my two cents and I could be completely off-base. :)
>>
>>
>>
>> Good luck!
>>
>>
>>
>> Ben
>>
>>
>>
>> Benjamin J. Chapman, J.D.
>> Assistant Dean for Information Technology, Emory University School of Law
>> Office: 404-727-6948 Cell/Text: 404-313-9544 IM: benjamin.chapman@gmail.com
>> Fewer emails in 2014: http://emailcharter.org
>>
>>
>> From: teknoids-bounces@ruckus.law.cornell.edu <teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <johnson@law.unm.edu>
>> Sent: Monday, February 10, 2014 1:47 PM
>> To: 'Teknoids'
>> Subject: [teknoids] Interpretation of goal
>>
>> Good morning everyone,
>> How would you interpret this performance review goal? “Focus on new classroom technologies with proven learning gain results”.
>> Or better yet, what classroom technologies has your school implemented that actually resulted in “learning gain”? Like many of you (I assume), we struggle with providing classroom bells and whistles that aren’t necessarily used. We’ve done faculty lunch and learns, and last year held a session where we asked faculty what they really want/need in classrooms. That did spark a couple of suggestions and we’re now replacing many classroom monitors with SMART monitors, but I’m not sure having the ability to mark up documents really results in increased learning.
>> Thoughts anyone?
>> Thanks…
>>
>> Cyndi Johnson
>> Director/Assistant Dean for Information Technology
>> UNM School of Law
>> (505) 277-0695
>>
>>
>> _______________________________________________
>> You are currently subscribed to teknoids as: sydbeckman@gmail.com.
>> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
>> --
>> See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.
>>
>>
>> _______________________________________________
>> You are currently subscribed to teknoids as: jmayer@cali.org.
>> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
>> --
>> See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.
>>
>>
>>

RE: Interpretation of goal

Rich,

"I was tempted to title one of the sections in document, "Education Technology - It's the Pedagogy Stupid!""

Double-dog dare you (or whatever is the Canadian equivalent)...

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf Of Rich McCue
Sent: Wednesday, February 12, 2014 1:57 PM
To: Teknoids
Subject: Re: [teknoids] Interpretation of goal

Below are a couple of paragraphs from a lit review I recently completed in preparation for some research I'm hoping to do this year on the effectiveness of using a flipped pedagogy in information literacy instruction. In short, my take on the literature is that new technologies don't make direct impact on student learning outcomes, however new technologies can enable new pedagogies that can have significant impacts on learning outcomes. I was tempted to title one of the sections in document, "Education Technology - It's the Pedagogy Stupid!" http://goo.gl/Iy9NMc

A Historical Perspective
When Clark (1994) defended his argument that the delivery media for instruction usually does not make a difference in, the technological landscape was much different. VHS video tapes were the primary means of watching on demand educational videos. Microsoft Windows 3.11 was the dominant desktop operating system, and dial-up modems using phone lines achieving speeds of 0.028 Mps were state of the art (compared to 2-20 Mps in 2013). At the same time, educational content was just starting to be distributed through relatively expensive multi-media CD-ROM applications for Windows and Macintosh. I suspect that in his era, Clark's assessment that technology does not improve instruction was close to 100% true. On the other hand, with 20+ years of maturation and significant improvements in bandwidth, hardware speed, and authoring tool usability improvements, technology is now in a position to make a large positive impact in the delivery of instruction by enabling new pedagogical approaches to instruction, like Flipped Classrooms and Problem Based Learning with virtual simulations and collaboration (Becker, 2010).
Pedagogy, Not Technology Key. While the passage of time has been kind to the pro-technology arguments of Kosma (1994), it is important to remember Clark (1994) was correct in arguing that no matter what new technology we use, if we do not also change pedagogy, the educational outcomes will stay the same (Oblinger & Hawkins, 2006). An example of this is a study that was conducted at the University of North Texas where a comparison of student retention of Information Literacy skills was measured between sections instructed in a traditional Face-to-Face class, a blended class, and an online class. In each of the three classes, the instructional materials and pedagogy were kept as uniform as possible. Not surprisingly, the researchers found that there was no significant difference in IL skills retention between the three different lecture delivery methods (Anderson & May, 2010).
In the large meta-study conducted by the US Department of education by Means et al. (2010), blended classes were found to have a statistically significantly higher summative assessment scores than FtF classes (2010). Given the lack of information about pedagogies used in the hundreds of studies they analysed, they stated that "the observed advantage for blended learning conditions is not necessarily rooted in the media user per se and may reflect differences in content, pedagogy and learning time" (p. xv). Happily for researchers, most post-2010 studies disclose more details about both media and pedagogies employed.
Typically, early in the adoption of new technologies, we do not take advantage of all the new capabilities available to us and tend to mimic activities that we are familiar with (Oblinger & Hawkins, 2006). A recent example of this is educators creating instructional materials for Massively Online Open Courses (MOOC) by videotaping complete FtF lectures and putting them online rather than taking advantage of the flexibility of the digital medium and dividing it into a number of shorter segments, adding extra audio and visual elements to enrich the instruction.

http://goo.gl/Iy9NMc
-----
Becker, K. (n.d.). The Clark-Kozma debate in the 21st century. Retrieved November 15, 2013, from https://www.academia.edu/462857/The_Clark-Kozma_Debate_in_the_21st_Century

Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29. doi:10.1007/BF02299088

Means, B., Yoyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. US Department of Education. Retrieved from http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalrepo...

Oblinger, D., & Hawkins, B. (2006). IT myths: The myth about no significant difference. Educause Review, 41(6), 14-15.

---
Rich McCue, BComm
University of Victoria Libraries
http://library.uvic.ca
http://richmccue.com

My PGP Public Key: http://msys.ca/PGPKeyRichMcCueUvicPublic.asc

On Feb 12, 2014, at 12:35 PM, Johnson, Cyndi <johnson@law.unm.edu> wrote:

It is an interesting topic. I think the instructional technologists on this list can provide more details than I can on how technology can enhance instruction and there's probably more research based around the impact of technology on learning outcomes than I want to dive into. Those are the proxies of measurement you reference, right?
My quick take-away is that students helping students is old news. Tutors, discussions, study groups, debates outside of class...nothing new there. What's interesting to me is the idea that technology can play a role in how students help students in ways I haven't thought of. Look at how people interact now. Sure, lots of sitting around and talking. But I'd guess - no, I know - that our students are connected to each other via Twitter, FB, Snapchat, etc etc etc. Why not use that interest in a way that can help them improve their analytical skills? Maybe not "learn more" or have a deeper understanding...that's too vague. But skills are measurable.
Cyndi

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfJohn Mayer
Sent: Wednesday, February 12, 2014 1:20 PM
To: Teknoids
Subject: Re: [teknoids] RE: Interpretation of goal

This is actually a very interesting question.

My first thought was what if we were talking about types of hammers. Would you ask if a particular type of hammer improves construction? Technology is almost never the point in education (perhaps I shouldn't hedge)...OK, TECHNOLOGY IS NOT THE GOAL.

So asking if a particular technology improved learning is the wrong question. First of all, learning is what happens inside the student's head and that's impossible to measure at any deep fidelity. At best, we use proxies like tests/assessments and interpersonal reactions ("does the student look confused?") to "measure" learning.

Sometimes, technology can reduce the administrative overhead and quite a bit. Before books, we had the oral tradition. Before screens, we had chalkboards. Before powerpoint, we had arm waving (Ok, now we have both).

It's easier to show a picture than draw it from scratch every time you teach the class (mostly) because technology.

It's easier to say "go away and look at this on the web" and have pretty good confidence that 15 or 30 or 150 people can do that ... because technology.

So those things are easier than the logistics of lumber, pulp, printing, shipping, book stores, credit cards and backpacks.

Now, once you radically change the availability of vast amounts of information, you change the way that teaching can happen and can only begin to explore if more/better learning is happening, and we are back to proxies of measurement.

I'm not helping here I think.

I would interpret this as HR wants an answer that demonstrates that you are thinking about the core goals and mission of the university/law school and that you are more than a brain stem punching buttons. Your desire to understand and answer this question shows exactly that. I would turn this into a discussion of the issues instead of trying to give a pat answer. There is no pat answer,so give them a buffet of answers.

John

On Wed, Feb 12, 2014 at 9:30 AM, Johnson, Cyndi <johnson@law.unm.edu> wrote:
Thanks Syd, Ben, and others who replied off list...
Ben was on track when he said the goal was vague. I took it to mean that I was to walk down the path that Syd mentions below but that wasn't the intention. What I'm asked to research is how students can use technology to improve their learning. For example, he mentioned research done by his colleague at Pitt, Kevin Ashley, that studied how use of a system called Comrade, used to provide anonymous feedback from other students, can improve learning gains. So I believe the goal has nothing to do with faculty's use of technology (or lack thereof) but, rather, how students can help other students. I think the study was focused primarily on the legal writing class but I haven't had time to look at it very hard.
Of course, it seems to me that feedback provided by students is just that - student feedback - and would only be as good (or bad) as the other students in the group.
Cyndi

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfSyd Beckman
Sent: Wednesday, February 12, 2014 8:15 AM
To: Teknoids
Subject: Re: [teknoids] RE: Interpretation of goal

Hi Ben and Cyndi and noids...

I don't necessarily think the goal confuses the two but - you would need a "study" to determine whether technology enhanced learning. Ben is right in that the subject would need defined learning outcomes and then - in theory - if you taught with and without the technology you could see of there was "gain." Of course, there is a huge problem. The teaching would have to be identical EXCEPT for the use of technology. Otherwise you have variables which are not controlled for and, as a result, you could not achieve statistically significant results.

So - in other words - teach Subject X using technology; Teach subject X WITHOUT the technology. Then compare the results. Then there is the variable of students. The only way to overcome that variable is many many data points. So you would have to conduct this study over years. And then - once you had your results - the technology would be so out of date that no one would want to use it. So you start over with new technology. :) Of course that is an exaggeration but you get the idea.

Syd Beckman
Professor of Law
LMU-Duncan School of Law

On Wed, Feb 12, 2014 at 9:19 AM, Chapman, Ben <ben.chapman@emory.edu> wrote:

Cyndi,

Sorry for the delayed response to this question - and my apologies if you've already gotten a slew of responses and I'm just repeating what others have said. :)

I don't know how to interpret this as a performance review goal and I think it would be frustrating to see this as a review goal. So, if I got this, rather than spending a lot of effort on trying to understand the goal as written, I would go back to the author and request clarification. The goal as written confuses pedagogy and technology. If the class itself has no clearly stated learning objectives or goals, I think it would be very difficult to measure whether the provided technology supports the (unstated) goals of the class.

In that discussion, I would probably reference something like this: www.academicpartnerships.com/docs/default-document-library/newbooklet15_singleb.pdf . See the section on the Quality Matters rubric on page 6 or so. It does the right thing, I think, by situating technology within the context of a broader attempt to establish standards for teaching. Of course, it's oriented towards online or blended teaching, but I think the list of items is directly applicable to technology-assisted in-class teaching.

Just my two cents and I could be completely off-base. :)

Good luck!

Ben

Benjamin J. Chapman, J.D.
Assistant Dean for Information Technology, Emory University School of Law
Office: 404-727-6948 Cell/Text: 404-313-9544 IM: benjamin.chapman@gmail.com
Fewer emails in 2014: http://emailcharter.org

________________________________
From: teknoids-bounces@ruckus.law.cornell.edu <teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <johnson@law.unm.edu>
Sent: Monday, February 10, 2014 1:47 PM
To: 'Teknoids'
Subject: [teknoids] Interpretation of goal

Good morning everyone,
How would you interpret this performance review goal? "Focus on new classroom technologies with proven learning gain results".
Or better yet, what classroom technologies has your school implemented that actually resulted in "learning gain"? Like many of you (I assume), we struggle with providing classroom bells and whistles that aren't necessarily used. We've done faculty lunch and learns, and last year held a session where we asked faculty what they really want/need in classrooms. That did spark a couple of suggestions and we're now replacing many classroom monitors with SMART monitors, but I'm not sure having the ability to mark up documents really results in increased learning.
Thoughts anyone?
Thanks...

Cyndi Johnson
Director/Assistant Dean for Information Technology
UNM School of Law
(505) 277-0695

_______________________________________________
You are currently subscribed to teknoids as: sydbeckman@gmail.com.
To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
--
See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.

_______________________________________________
You are currently subscribed to teknoids as: jmayer@cali.org.
To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
--
See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.

--
----------
John Mayer
Executive Director
Center for Computer-Assisted Legal Instruction/CALI
565 West Adams
Chicago, IL 60661
312-906-5307
312-906-5280 - fax
jmayer@cali.org
http://www.cali.org
twitter.com/johnpmayer
----------
_______________________________________________
You are currently subscribed to teknoids as: rmccue@uvic.ca.
To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
--
See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.

Interpretation of goal

Maybe the title for a conference presentation ;-)
---
Rich McCue, BComm
University of Victoria Libraries
http://library.uvic.ca
http://richmccue.com

My PGP Public Key: http://msys.ca/PGPKeyRichMcCueUvicPublic.asc

On Feb 12, 2014, at 1:16 PM, Johnson, Cyndi <johnson@law.unm.edu> wrote:

> Rich,
>
> “I was tempted to title one of the sections in document, “Education Technology - It’s the Pedagogy Stupid!"”
>
> Double-dog dare you (or whatever is the Canadian equivalent)…
>
> Cyndi Johnson
> Director/Assistant Dean for Information Technology
> UNM School of Law
> (505) 277-0695
>
> From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfRich McCue
> Sent: Wednesday, February 12, 2014 1:57 PM
> To: Teknoids
> Subject: Re: [teknoids] Interpretation of goal
>
> Below are a couple of paragraphs from a lit review I recently completed in preparation for some research I’m hoping to do this year on the effectiveness of using a flipped pedagogy in information literacy instruction. In short, my take on the literature is that new technologies don't make direct impact on student learning outcomes, however new technologies can enable new pedagogies that can have significant impacts on learning outcomes. I was tempted to title one of the sections in document, “Education Technology - It’s the Pedagogy Stupid!" http://goo.gl/Iy9NMc
>
> A Historical Perspective
> When Clark (1994) defended his argument that the delivery media for instruction usually does not make a difference in, the technological landscape was much different. VHS video tapes were the primary means of watching on demand educational videos. Microsoft Windows 3.11 was the dominant desktop operating system, and dial-up modems using phone lines achieving speeds of 0.028 Mps were state of the art (compared to 2-20 Mps in 2013). At the same time, educational content was just starting to be distributed through relatively expensive multi-media CD-ROM applications for Windows and Macintosh. I suspect that in his era, Clark’s assessment that technology does not improve instruction was close to 100% true. On the other hand, with 20+ years of maturation and significant improvements in bandwidth, hardware speed, and authoring tool usability improvements, technology is now in a position to make a large positive impact in the delivery of instruction by enabling new pedagogical approaches to instruction, like Flipped Classrooms and Problem Based Learning with virtual simulations and collaboration (Becker, 2010).
> Pedagogy, Not Technology Key. While the passage of time has been kind to the pro-technology arguments of Kosma (1994), it is important to remember Clark (1994) was correct in arguing that no matter what new technology we use, if we do not also change pedagogy, the educational outcomes will stay the same (Oblinger & Hawkins, 2006). An example of this is a study that was conducted at the University of North Texas where a comparison of student retention of Information Literacy skills was measured between sections instructed in a traditional Face-to-Face class, a blended class, and an online class. In each of the three classes, the instructional materials and pedagogy were kept as uniform as possible. Not surprisingly, the researchers found that there was no significant difference in IL skills retention between the three different lecture delivery methods (Anderson & May, 2010).
> In the large meta-study conducted by the US Department of education by Means et al. (2010), blended classes were found to have a statistically significantly higher summative assessment scores than FtF classes (2010). Given the lack of information about pedagogies used in the hundreds of studies they analysed, they stated that “the observed advantage for blended learning conditions is not necessarily rooted in the media user per se and may reflect differences in content, pedagogy and learning time” (p. xv). Happily for researchers, most post-2010 studies disclose more details about both media and pedagogies employed.
> Typically, early in the adoption of new technologies, we do not take advantage of all the new capabilities available to us and tend to mimic activities that we are familiar with (Oblinger & Hawkins, 2006). A recent example of this is educators creating instructional materials for Massively Online Open Courses (MOOC) by videotaping complete FtF lectures and putting them online rather than taking advantage of the flexibility of the digital medium and dividing it into a number of shorter segments, adding extra audio and visual elements to enrich the instruction.
>
> http://goo.gl/Iy9NMc
> -----
> Becker, K. (n.d.). The Clark-Kozma debate in the 21st century. Retrieved November 15, 2013, from https://www.academia.edu/462857/The_Clark-Kozma_Debate_in_the_21st_Century
>
> Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29. doi:10.1007/BF02299088
>
> Means, B., Yoyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. US Department of Education. Retrieved from http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalrepo...
>
> Oblinger, D., & Hawkins, B. (2006). IT myths: The myth about no significant difference. Educause Review, 41(6), 14–15.
>
>
>
>
>
> ---
> Rich McCue, BComm
> University of Victoria Libraries
> http://library.uvic.ca
> http://richmccue.com
>
> My PGP Public Key: http://msys.ca/PGPKeyRichMcCueUvicPublic.asc
>
>
> On Feb 12, 2014, at 12:35 PM, Johnson, Cyndi <johnson@law.unm.edu> wrote:
>
>
> It is an interesting topic. I think the instructional technologists on this list can provide more details than I can on how technology can enhance instruction and there’s probably more research based around the impact of technology on learning outcomes than I want to dive into. Those are the proxies of measurement you reference, right?
> My quick take-away is that students helping students is old news. Tutors, discussions, study groups, debates outside of class…nothing new there. What’s interesting to me is the idea that technology can play a role in how students help students in ways I haven’t thought of. Look at how people interact now. Sure, lots of sitting around and talking. But I’d guess – no, I know – that our students are connected to each other via Twitter, FB, Snapchat, etc etc etc. Why not use that interest in a way that can help them improve their analytical skills? Maybe not “learn more” or have a deeper understanding…that’s too vague. But skills are measurable.
> Cyndi
>
> Cyndi Johnson
> Director/Assistant Dean for Information Technology
> UNM School of Law
> (505) 277-0695
>
> From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfJohn Mayer
> Sent: Wednesday, February 12, 2014 1:20 PM
> To: Teknoids
> Subject: Re: [teknoids] RE: Interpretation of goal
>
> This is actually a very interesting question.
>
> My first thought was what if we were talking about types of hammers. Would you ask if a particular type of hammer improves construction? Technology is almost never the point in education (perhaps I shouldn't hedge)...OK, TECHNOLOGY IS NOT THE GOAL.
>
> So asking if a particular technology improved learning is the wrong question. First of all, learning is what happens inside the student's head and that's impossible to measure at any deep fidelity. At best, we use proxies like tests/assessments and interpersonal reactions ("does the student look confused?") to "measure" learning.
>
> Sometimes, technology can reduce the administrative overhead and quite a bit. Before books, we had the oral tradition. Before screens, we had chalkboards. Before powerpoint, we had arm waving (Ok, now we have both).
>
> It's easier to show a picture than draw it from scratch every time you teach the class (mostly) because technology.
>
> It's easier to say "go away and look at this on the web" and have pretty good confidence that 15 or 30 or 150 people can do that ... because technology.
>
> So those things are easier than the logistics of lumber, pulp, printing, shipping, book stores, credit cards and backpacks.
>
> Now, once you radically change the availability of vast amounts of information, you change the way that teaching can happen and can only begin to explore if more/better learning is happening, and we are back to proxies of measurement.
>
> I'm not helping here I think.
>
> I would interpret this as HR wants an answer that demonstrates that you are thinking about the core goals and mission of the university/law school and that you are more than a brain stem punching buttons. Your desire to understand and answer this question shows exactly that. I would turn this into a discussion of the issues instead of trying to give a pat answer. There is no pat answer,so give them a buffet of answers.
>
> John
>
>
>
>
>
>
> On Wed, Feb 12, 2014 at 9:30 AM, Johnson, Cyndi <johnson@law.unm.edu> wrote:
> Thanks Syd, Ben, and others who replied off list…
> Ben was on track when he said the goal was vague. I took it to mean that I was to walk down the path that Syd mentions below but that wasn’t the intention. What I’m asked to research is how students can use technology to improve their learning. For example, he mentioned research done by his colleague at Pitt, Kevin Ashley, that studied how use of a system called Comrade, used to provide anonymous feedback from other students, can improve learning gains. So I believe the goal has nothing to do with faculty’s use of technology (or lack thereof) but, rather, how students can help other students. I think the study was focused primarily on the legal writing class but I haven’t had time to look at it very hard.
> Of course, it seems to me that feedback provided by students is just that – student feedback - and would only be as good (or bad) as the other students in the group.
> Cyndi
>
> Cyndi Johnson
> Director/Assistant Dean for Information Technology
> UNM School of Law
> (505) 277-0695
>
> From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfSyd Beckman
> Sent: Wednesday, February 12, 2014 8:15 AM
> To: Teknoids
> Subject: Re: [teknoids] RE: Interpretation of goal
>
> Hi Ben and Cyndi and noids...
>
> I don't necessarily think the goal confuses the two but - you would need a "study" to determine whether technology enhanced learning. Ben is right in that the subject would need defined learning outcomes and then - in theory - if you taught with and without the technology you could see of there was "gain." Of course, there is a huge problem. The teaching would have to be identical EXCEPT for the use of technology. Otherwise you have variables which are not controlled for and, as a result, you could not achieve statistically significant results.
>
> So - in other words - teach Subject X using technology; Teach subject X WITHOUT the technology. Then compare the results. Then there is the variable of students. The only way to overcome that variable is many many data points. So you would have to conduct this study over years. And then - once you had your results - the technology would be so out of date that no one would want to use it. So you start over with new technology. :) Of course that is an exaggeration but you get the idea.
>
> Syd Beckman
> Professor of Law
> LMU-Duncan School of Law
>
>
>
> On Wed, Feb 12, 2014 at 9:19 AM, Chapman, Ben <ben.chapman@emory.edu> wrote:
> Cyndi,
>
>
>
> Sorry for the delayed response to this question - and my apologies if you've already gotten a slew of responses and I'm just repeating what others have said. :)
>
>
>
> I don't know how to interpret this as a performance review goal and I think it would be frustrating to see this as a review goal. So, if I got this, rather than spending a lot of effort on trying to understand the goal as written, I would go back to the author and request clarification. The goal as written confuses pedagogy and technology. If the class itself has no clearly stated learning objectives or goals, I think it would be very difficult to measure whether the provided technology supports the (unstated) goals of the class.
>
>
>
> In that discussion, I would probably reference something like this: www.academicpartnerships.com/docs/default-document-library/newbooklet15_... . See the section on the Quality Matters rubric on page 6 or so. It does the right thing, I think, by situating technology within the context of a broader attempt to establish standards for teaching. Of course, it's oriented towards online or blended teaching, but I think the list of items is directly applicable to technology-assisted in-class teaching.
>
>
>
> Just my two cents and I could be completely off-base. :)
>
>
>
> Good luck!
>
>
>
> Ben
>
>
>
> Benjamin J. Chapman, J.D.
> Assistant Dean for Information Technology, Emory University School of Law
> Office: 404-727-6948 Cell/Text: 404-313-9544 IM: benjamin.chapman@gmail.com
> Fewer emails in 2014: http://emailcharter.org
>
>
> From: teknoids-bounces@ruckus.law.cornell.edu <teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <johnson@law.unm.edu>
> Sent: Monday, February 10, 2014 1:47 PM
> To: 'Teknoids'
> Subject: [teknoids] Interpretation of goal
>
> Good morning everyone,
> How would you interpret this performance review goal? “Focus on new classroom technologies with proven learning gain results”.
> Or better yet, what classroom technologies has your school implemented that actually resulted in “learning gain”? Like many of you (I assume), we struggle with providing classroom bells and whistles that aren’t necessarily used. We’ve done faculty lunch and learns, and last year held a session where we asked faculty what they really want/need in classrooms. That did spark a couple of suggestions and we’re now replacing many classroom monitors with SMART monitors, but I’m not sure having the ability to mark up documents really results in increased learning.
> Thoughts anyone?
> Thanks…
>
> Cyndi Johnson
> Director/Assistant Dean for Information Technology
> UNM School of Law
> (505) 277-0695
>
>
> _______________________________________________
> You are currently subscribed to teknoids as: sydbeckman@gmail.com.
> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
> --
> See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.
>
>
> _______________________________________________
> You are currently subscribed to teknoids as: jmayer@cali.org.
> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
> --
> See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.
>
>
>

Interpretation of goal

One that I think many would be interested in!

Also, Courtney Selby alerted me that the link that I provided in my original post no longer works - here's one that does work:

http://www.academicpartnerships.com/sites/default/files/A%20Guide%20to%2...

It's CC-licensed; although the material that it cites and quotes is not.

Thanks, Courtney!

Ben

On Feb 12, 2014, at 7:00 PM, Rich McCue <rmccue@uvic.ca> wrote:

> Maybe the title for a conference presentation ;-)
> ---
> Rich McCue, BComm
> University of Victoria Libraries
> http://library.uvic.ca
> http://richmccue.com
>
> My PGP Public Key: http://msys.ca/PGPKeyRichMcCueUvicPublic.asc
>
>
> On Feb 12, 2014, at 1:16 PM, Johnson, Cyndi <johnson@law.unm.edu> wrote:
>
>> Rich,
>>
>> “I was tempted to title one of the sections in document, “Education Technology - It’s the Pedagogy Stupid!"”
>>
>> Double-dog dare you (or whatever is the Canadian equivalent)…
>>
>> Cyndi Johnson
>> Director/Assistant Dean for Information Technology
>> UNM School of Law
>> (505) 277-0695
>>
>> From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfRich McCue
>> Sent: Wednesday, February 12, 2014 1:57 PM
>> To: Teknoids
>> Subject: Re: [teknoids] Interpretation of goal
>>
>> Below are a couple of paragraphs from a lit review I recently completed in preparation for some research I’m hoping to do this year on the effectiveness of using a flipped pedagogy in information literacy instruction. In short, my take on the literature is that new technologies don't make direct impact on student learning outcomes, however new technologies can enable new pedagogies that can have significant impacts on learning outcomes. I was tempted to title one of the sections in document, “Education Technology - It’s the Pedagogy Stupid!" http://goo.gl/Iy9NMc
>>
>> A Historical Perspective
>> When Clark (1994) defended his argument that the delivery media for instruction usually does not make a difference in, the technological landscape was much different. VHS video tapes were the primary means of watching on demand educational videos. Microsoft Windows 3.11 was the dominant desktop operating system, and dial-up modems using phone lines achieving speeds of 0.028 Mps were state of the art (compared to 2-20 Mps in 2013). At the same time, educational content was just starting to be distributed through relatively expensive multi-media CD-ROM applications for Windows and Macintosh. I suspect that in his era, Clark’s assessment that technology does not improve instruction was close to 100% true. On the other hand, with 20+ years of maturation and significant improvements in bandwidth, hardware speed, and authoring tool usability improvements, technology is now in a position to make a large positive impact in the delivery of instruction by enabling new pedagogical approaches to instruction, like Flipped Classrooms and Problem Based Learning with virtual simulations and collaboration (Becker, 2010).
>> Pedagogy, Not Technology Key. While the passage of time has been kind to the pro-technology arguments of Kosma (1994), it is important to remember Clark (1994) was correct in arguing that no matter what new technology we use, if we do not also change pedagogy, the educational outcomes will stay the same (Oblinger & Hawkins, 2006). An example of this is a study that was conducted at the University of North Texas where a comparison of student retention of Information Literacy skills was measured between sections instructed in a traditional Face-to-Face class, a blended class, and an online class. In each of the three classes, the instructional materials and pedagogy were kept as uniform as possible. Not surprisingly, the researchers found that there was no significant difference in IL skills retention between the three different lecture delivery methods (Anderson & May, 2010).
>> In the large meta-study conducted by the US Department of education by Means et al. (2010), blended classes were found to have a statistically significantly higher summative assessment scores than FtF classes (2010). Given the lack of information about pedagogies used in the hundreds of studies they analysed, they stated that “the observed advantage for blended learning conditions is not necessarily rooted in the media user per se and may reflect differences in content, pedagogy and learning time” (p. xv). Happily for researchers, most post-2010 studies disclose more details about both media and pedagogies employed.
>> Typically, early in the adoption of new technologies, we do not take advantage of all the new capabilities available to us and tend to mimic activities that we are familiar with (Oblinger & Hawkins, 2006). A recent example of this is educators creating instructional materials for Massively Online Open Courses (MOOC) by videotaping complete FtF lectures and putting them online rather than taking advantage of the flexibility of the digital medium and dividing it into a number of shorter segments, adding extra audio and visual elements to enrich the instruction.
>>
>> http://goo.gl/Iy9NMc
>> -----
>> Becker, K. (n.d.). The Clark-Kozma debate in the 21st century. Retrieved November 15, 2013, from https://www.academia.edu/462857/The_Clark-Kozma_Debate_in_the_21st_Century
>>
>> Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29. doi:10.1007/BF02299088
>>
>> Means, B., Yoyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. US Department of Education. Retrieved from http://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalrepo...
>>
>> Oblinger, D., & Hawkins, B. (2006). IT myths: The myth about no significant difference. Educause Review, 41(6), 14–15.
>>
>>
>>
>>
>>
>> ---
>> Rich McCue, BComm
>> University of Victoria Libraries
>> http://library.uvic.ca
>> http://richmccue.com
>>
>> My PGP Public Key: http://msys.ca/PGPKeyRichMcCueUvicPublic.asc
>>
>>
>> On Feb 12, 2014, at 12:35 PM, Johnson, Cyndi <johnson@law.unm.edu> wrote:
>>
>>
>> It is an interesting topic. I think the instructional technologists on this list can provide more details than I can on how technology can enhance instruction and there’s probably more research based around the impact of technology on learning outcomes than I want to dive into. Those are the proxies of measurement you reference, right?
>> My quick take-away is that students helping students is old news. Tutors, discussions, study groups, debates outside of class…nothing new there. What’s interesting to me is the idea that technology can play a role in how students help students in ways I haven’t thought of. Look at how people interact now. Sure, lots of sitting around and talking. But I’d guess – no, I know – that our students are connected to each other via Twitter, FB, Snapchat, etc etc etc. Why not use that interest in a way that can help them improve their analytical skills? Maybe not “learn more” or have a deeper understanding…that’s too vague. But skills are measurable.
>> Cyndi
>>
>> Cyndi Johnson
>> Director/Assistant Dean for Information Technology
>> UNM School of Law
>> (505) 277-0695
>>
>> From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfJohn Mayer
>> Sent: Wednesday, February 12, 2014 1:20 PM
>> To: Teknoids
>> Subject: Re: [teknoids] RE: Interpretation of goal
>>
>> This is actually a very interesting question.
>>
>> My first thought was what if we were talking about types of hammers. Would you ask if a particular type of hammer improves construction? Technology is almost never the point in education (perhaps I shouldn't hedge)...OK, TECHNOLOGY IS NOT THE GOAL.
>>
>> So asking if a particular technology improved learning is the wrong question. First of all, learning is what happens inside the student's head and that's impossible to measure at any deep fidelity. At best, we use proxies like tests/assessments and interpersonal reactions ("does the student look confused?") to "measure" learning.
>>
>> Sometimes, technology can reduce the administrative overhead and quite a bit. Before books, we had the oral tradition. Before screens, we had chalkboards. Before powerpoint, we had arm waving (Ok, now we have both).
>>
>> It's easier to show a picture than draw it from scratch every time you teach the class (mostly) because technology.
>>
>> It's easier to say "go away and look at this on the web" and have pretty good confidence that 15 or 30 or 150 people can do that ... because technology.
>>
>> So those things are easier than the logistics of lumber, pulp, printing, shipping, book stores, credit cards and backpacks.
>>
>> Now, once you radically change the availability of vast amounts of information, you change the way that teaching can happen and can only begin to explore if more/better learning is happening, and we are back to proxies of measurement.
>>
>> I'm not helping here I think.
>>
>> I would interpret this as HR wants an answer that demonstrates that you are thinking about the core goals and mission of the university/law school and that you are more than a brain stem punching buttons. Your desire to understand and answer this question shows exactly that. I would turn this into a discussion of the issues instead of trying to give a pat answer. There is no pat answer,so give them a buffet of answers.
>>
>> John
>>
>>
>>
>>
>>
>>
>> On Wed, Feb 12, 2014 at 9:30 AM, Johnson, Cyndi <johnson@law.unm.edu> wrote:
>> Thanks Syd, Ben, and others who replied off list…
>> Ben was on track when he said the goal was vague. I took it to mean that I was to walk down the path that Syd mentions below but that wasn’t the intention. What I’m asked to research is how students can use technology to improve their learning. For example, he mentioned research done by his colleague at Pitt, Kevin Ashley, that studied how use of a system called Comrade, used to provide anonymous feedback from other students, can improve learning gains. So I believe the goal has nothing to do with faculty’s use of technology (or lack thereof) but, rather, how students can help other students. I think the study was focused primarily on the legal writing class but I haven’t had time to look at it very hard.
>> Of course, it seems to me that feedback provided by students is just that – student feedback - and would only be as good (or bad) as the other students in the group.
>> Cyndi
>>
>> Cyndi Johnson
>> Director/Assistant Dean for Information Technology
>> UNM School of Law
>> (505) 277-0695
>>
>> From: teknoids-bounces@ruckus.law.cornell.edu [mailto:teknoids-bounces@ruckus.law.cornell.edu] On Behalf OfSyd Beckman
>> Sent: Wednesday, February 12, 2014 8:15 AM
>> To: Teknoids
>> Subject: Re: [teknoids] RE: Interpretation of goal
>>
>> Hi Ben and Cyndi and noids...
>>
>> I don't necessarily think the goal confuses the two but - you would need a "study" to determine whether technology enhanced learning. Ben is right in that the subject would need defined learning outcomes and then - in theory - if you taught with and without the technology you could see of there was "gain." Of course, there is a huge problem. The teaching would have to be identical EXCEPT for the use of technology. Otherwise you have variables which are not controlled for and, as a result, you could not achieve statistically significant results.
>>
>> So - in other words - teach Subject X using technology; Teach subject X WITHOUT the technology. Then compare the results. Then there is the variable of students. The only way to overcome that variable is many many data points. So you would have to conduct this study over years. And then - once you had your results - the technology would be so out of date that no one would want to use it. So you start over with new technology. :) Of course that is an exaggeration but you get the idea.
>>
>> Syd Beckman
>> Professor of Law
>> LMU-Duncan School of Law
>>
>>
>>
>> On Wed, Feb 12, 2014 at 9:19 AM, Chapman, Ben <ben.chapman@emory.edu> wrote:
>> Cyndi,
>>
>>
>>
>> Sorry for the delayed response to this question - and my apologies if you've already gotten a slew of responses and I'm just repeating what others have said. :)
>>
>>
>>
>> I don't know how to interpret this as a performance review goal and I think it would be frustrating to see this as a review goal. So, if I got this, rather than spending a lot of effort on trying to understand the goal as written, I would go back to the author and request clarification. The goal as written confuses pedagogy and technology. If the class itself has no clearly stated learning objectives or goals, I think it would be very difficult to measure whether the provided technology supports the (unstated) goals of the class.
>>
>>
>>
>> In that discussion, I would probably reference something like this: www.academicpartnerships.com/docs/default-document-library/newbooklet15_... . See the section on the Quality Matters rubric on page 6 or so. It does the right thing, I think, by situating technology within the context of a broader attempt to establish standards for teaching. Of course, it's oriented towards online or blended teaching, but I think the list of items is directly applicable to technology-assisted in-class teaching.
>>
>>
>>
>> Just my two cents and I could be completely off-base. :)
>>
>>
>>
>> Good luck!
>>
>>
>>
>> Ben
>>
>>
>>
>> Benjamin J. Chapman, J.D.
>> Assistant Dean for Information Technology, Emory University School of Law
>> Office: 404-727-6948 Cell/Text: 404-313-9544 IM: benjamin.chapman@gmail.com
>> Fewer emails in 2014: http://emailcharter.org
>>
>>
>> From: teknoids-bounces@ruckus.law.cornell.edu <teknoids-bounces@ruckus.law.cornell.edu> on behalf of Johnson, Cyndi <johnson@law.unm.edu>
>> Sent: Monday, February 10, 2014 1:47 PM
>> To: 'Teknoids'
>> Subject: [teknoids] Interpretation of goal
>>
>> Good morning everyone,
>> How would you interpret this performance review goal? “Focus on new classroom technologies with proven learning gain results”.
>> Or better yet, what classroom technologies has your school implemented that actually resulted in “learning gain”? Like many of you (I assume), we struggle with providing classroom bells and whistles that aren’t necessarily used. We’ve done faculty lunch and learns, and last year held a session where we asked faculty what they really want/need in classrooms. That did spark a couple of suggestions and we’re now replacing many classroom monitors with SMART monitors, but I’m not sure having the ability to mark up documents really results in increased learning.
>> Thoughts anyone?
>> Thanks…
>>
>> Cyndi Johnson
>> Director/Assistant Dean for Information Technology
>> UNM School of Law
>> (505) 277-0695
>>
>>
>> _______________________________________________
>> You are currently subscribed to teknoids as: sydbeckman@gmail.com.
>> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
>> --
>> See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.
>>
>>
>> _______________________________________________
>> You are currently subscribed to teknoids as: jmayer@cali.org.
>> To unsubscribe send a blank email to teknoids-leave@ruckus.law.cornell.edu
>> --
>> See the web interface at http://ruckus.law.cornell.edu/mailman/listinfo/teknoids to get your list password, unsubscribe, and view your list settings.
>>
>>
>>

Ben Chapman
Assistant Dean, Information Technology
Emory University School of Law