(Sample 1)
Marsha is on her way. She called from the car phone I think. It sounded like the car phone, to let us know that she would be delayed.
I would like to welcome two people who haven't been with us before.
Suzanne Clewell, we're delighted to have you with us today. Suzanne, would you tell us a little bit about what you do?
It's great to be here.
Judith.
I have carefully read and heard about all of the things that the group has discussed up until now.
I'm a Professor of Education at the University of Albany, the State University of New York. And I'm also the Director of the National Research Center on English Learning and Achievement.
(Laughter)
And we have a new member, Gloria Lopez Gutierrez.
And, Gloria, tell us a little bit about yourself.
I'll check what the population is there. Our student population, I don't know, but it's certainly under 10,000. I would say something like 7,000 or 8,000, a small district.
I'm a third grade teacher. I'm a parent of a soon-to-be fourth grade student.
(Laughter)
David, some opening remarks.
(Laughter)
And many thanks to Eunice and Matt for lots of hard work and for really making a big leap forward and for going from where we were in Chicago in terms of text to where we are today.
And there was a real hard attempt, both to be faithful to the conversation that took place in Chicago and to capture the good ideas around the table and put them into some logical order that would be reader friendly, and also to put forward a couple of ideas about how to address the set of issues that have been around the table, but may not have been resolved.
And so some of what's in front of you is a bit perspective and anticipatory about where you might be going and at least to sort of test the ground to see if we've got it recorded right or if there's a better way to do it or a different way to do it.
Also, I noted that the passage of this document from our office to you in some cases was not as smooth as it might have been and including the hotel conspiring against us last night to not deliver this document to Eunice and to Dorothy among others.
(Laughter)
It's also the case that in reading over the summary of the last meetings which is under tab F, I believe, we discovered that we had such a good summary of the meeting that there are some things in the summary that haven't been yet transferred to the document.
And we are prepared to tell you what those things are. That is, things that you decided that were well captured in the summary, but haven't yet all made it into the document, but that need to be.
I don't know if you noticed their absence or not, but we did. And we want to correct that.
In addition, you have before you here in the document a set of new text that Eunice found, searched for this past week in between writing, and a set of prospective questions that one might ask around this.
And Eunice might want to say a little bit about what that is and what they are and how they came to be.
And then, I would just like in closing to say that we've got a good deal of work ahead of us, even though we got a lot accomplished.
We have a National Test Panel meeting coming up on Wednesday and Thursday of this week, where Dorothy will basically provide the status report to the Test Panel about where things stand and share with them this draft or something very close to it.
Our objective is, I would say in the next 10 days or so to find a way to get a draft out to the public.
That is to take the current draft before you and refine it. And in that time -- I know you're going to refine it based on this conversation over the next two days.
But in that time, to get another draft to you to review so that the committee is clearly comfortable and ready to have its name put on the draft as a draft seeking public comments and critique.
And then, that draft will be the basis for hearings in Atlanta and the public comment process that we want to begin so that when you come to Atlanta in the middle of August, you will have a range of advice from people in the reading community, people in the Board of Education community, from folks in the measurement community about the merits of the work that has been done to date.
And that if we do all that well with the help and participation from all of you, we will get a lot of good advice that we can use and take advantage of.
We may get some advice that we are not happy about, but that is part of the process.
And you all have the good judgment to decide which advice is helpful and which may be less helpful.
And we are confident that you can do that.
I should also say on the comment process though, our intention is both to put up the draft on a Web site so that people have access to it that way, to make it available through e-mail to people who are on various lists, to mail it to the folks who were recommended to us to do that, and to, in fact, give it to anyone who wants it.
We all know that each of you have your own networks and a collection of colleagues, teachers, and scholars in the field.
And to the extent that you have those kinds of individuals whom we might otherwise miss that we sort of welcome you to provide us either e-mail addresses or regular plain old post office addresses so that we can get this out.
It's the case that if anyone wants to distribute it themselves, that's fine, too.
That would help the process if you could let us know basically where it's going so we can sort of track who's getting it.
And we can make sure that it gets distributed in a broad range of communities. Then, we get a complete diversity of responses in the review process.
And then, based on your review, there will be a second draft. And then, Dorothy and Marsh‡ will estimate the final call on it. And that will be what gets distributed.
But as soon as that is available, we will make it available to you in both hard and electronic versions to do as you see fit.
(Laughter)
(Laughter)
(Laughter)
And I know Wayne would be with us today, except he married off a daughter this weekend. And so he's flying back from Colorado today and joining us tomorrow.
And I imagine Gary will show up at some point.
Marsh‡ called. Marsh‡ had car problems. She's due in sooner or later. She was planning to be here on time. I talked to her yesterday.
You're the next person, Barbara. Surprise.
(Laughter)
(Laughter)
Are you going to talk about the Technical Committee phone call, David, in terms of an update? Or do you want me to start it?
(Laughter)
And they were pleased to hear that we are now up to six.
(Laughter)
But they also said that didn't remove the general interest and concern with the question. They thought it was important to do the kind of analysis that they had originally suggested that we do. And so that will happen.
And also we had a conversation with them about untimed tests where they were, I think, of two different minds.
One finds it attractive and for the reasons that people generally find it attractive.
That is, I guess mostly to remove the test anxiety and also to make sure that everyone has a full opportunity. And this is limited by the length of the test, the competence.
But at the same time, understanding that it was one of several ways in which the comparability issues with NAEP and our interest in linking this test and providing students and their teachers with some of their results in terms of where they are on the NAEP scale could be a compromise, not the only place, but one of several places.
And so they wanted us to sort of keep watching that and keep attending to it and in the course of pilot and field testing to think about possibly adding a little bunch of time and to also interview students about whether they thought they had enough time to complete.
I don't know that that exactly fully addresses the concern.
When this conversation was discussed in the Mathematics Committee, Wayne, amongst others but not alone, worried a lot about the sort of logistics and administrative complications that could ensue from an untimed test, to say nothing about the comparability issues.
And I think it was in part a concern that given this is supposedly a voluntary test -- it is a voluntary test that this might make the examination less attractive in some quarters than in others than it might otherwise be.
So I think -- is that close, Steve?
(Laughter)
Wayne weighs in pretty strongly on timed. The interesting thing is Wayne brings kind of the perspective of a state assessment director to this and all that you have to go through if you have an untimed test and the testing burden on that.
So that, you know, with Marsh‡ not here right now at least, it is interesting to kind of keep that perspective in mind because they've been through some of this.
The second thing is that an Ina was on the --
And one of the things that she indicated a concern about -- and it's just to keep in the back of your mind. I mean, she just said think about this.
And that is if you dramatically change the length of the passages -- the length of the passages that the kids are dealing with in this assessment, the degree to which it links back to NAEP is decreased.
So just -- that's something to keep in mind that what we say about what kids do on NAEP is based on certain kinds of passages.
And she just sort of gave us a caveat to sort of think about that as you do that.
And I know you have the range and everything, but just keep that in the back of your mind.
I don't know. Judith, do you want to weigh in on that?
And NAEP tries to have shorter and longer passages, as well. So I think that comparability should be able to hold in this.
The other thing that I just wanted to mention very quickly, because again I don't know what the concern was here among the people on this panel, but NAEP had done a very interesting sub-study with the Southern Regional Education Board, SREB.
I can't recall when, but it was for the writing assessment.
And one of the concerns at that point is what happens if students have more time?
And so in that assessment, it was a comparison of 20 minutes and 15 minutes to see if in fact the additional time made a difference.
I'm mentioning this here not because it's writing, but people might have been concerned about how much time was given to the students to read and then write the extended passage.
And they found that with the additional time, it made almost no difference.
The students who were least able to make any additional scoring advantage with the additional time were the students who were in fact the lower scorers anyway.
And the students who were the higher scorers did ever so much better.
Now, this is 20 minutes to 15 minutes. It's not a very insignificant amount of time, but 20 that minutes would be sufficient time.
This was -- I honestly don't know. I assume this was an eighth grade study instead of a fourth grade study.
So I just think we need to take that into account, too, when we are discussing time.
Some schools, because of logistics, are just going to cut it off and send the kids onto their next class. They are not going to give the kids as much time as they need.
Whereas, other sites may really take advantage of the opportunity.
So it just really increases the standardization. And that was another concern.
And there, the issue of comparability came up in sort of -- I won't say a positive, but in a supportive way saying that that would be the second item that would contribute to making it more comparable and that it also had a number of other positive properties that had to do with both the nature of what we expect kids to be able to do or we'll look for them to do in both reading and mathematics and that you can get at that better with those items.
Barbara also made the point at the meeting. And it sticks with me. And that is the modest difference in both what kids write in between a short constructed item and extended item and what that means in terms of scoring burden which are the big issues here.
And that, in fact, if you didn't have this second extended constructed item, you wouldn't have nothing, but you would have something else that would take time.
And so it does not say that the issue of what this might contribute to in the overall turnaround time and the scoring burden on the whole process isn't one that needs to be looked at hard, but that there are a range of factors stacking up in favor of the second extended constructed response item in both cases.
I think we got a very clear set of signals on that score and continuing encouragement for Steve's production of scenarios, multiple scenarios that would vary by number of students participating and what the implications of that would be for scoring burden and time and potential implications on the quality of scoring that could be achieved.
I really would like for this committee to make that decision about whether or not we want to have an untimed test or not.
I think it can be something that would be researched indeed, but I would like for us to make that decision and indeed any kind of accommodations.
I think we need to be very specific about what we want. And they need to be standardized for the very reasons that Naomi was addressing.
And then, Barb, we'll come back to you.
Because I understand the need for consistency, and I think that gives us some consistency, but it also says that we've paid attention to it and to our best knowledge, this is not a speeded test.
Barbara.
I mean, I don't think that we're stuck with the opposite ends of the continuum or whatever, 90 minutes versus untimed.
But, you know, it could be that based on pilot information, we may expand the time, but not appreciably.
But if you keep sort of a given set of exercises constant and it looks like maybe it takes another 15 minutes a day, that might be a more reasonable thing than thinking about untimed.
And the other thing though that I would want to take a look at in the pilot in terms of this timing thing is whether -- I mean, I like that 95 percent number, except that what we need to do is look at whether it's particularly difficult for certain groups of students, whether it's LEP students or students with special accommodations.
If that timing particularly biases against their performing well or their ability to demonstrate what they know, then I think that we would need to look at it also.
So I mean more than just the 95 percent. We need to look at the special populations that are usually impacted by time.
It seems to me that that's exactly the kind of data that we would need.
And then, you also look at the kinds of complications that occur when you do administer it in an untimed fashion.
And whatever you administer, the conditions you administer that form under are going to be the ones that have to be actually used.
So you're not going to be able to play with the intact form per se. You can take pieces and some separate kinds of studies.
But an intact, I don't see how you could do it.
I think there are some opportunities because the field test is supposed to be a process. We're going to give a whole form to each kid.
And then Marilyn.
We should be able to find something to look at that.
And so I would prefer to see or I would suggest that you consider seeing this as a research study to be done on the side.
And make a choice now. Go with it now with the understanding that we can do a research study around it.
It could be in the first couple of years. But we need item statistics that we can count on to do all the scaling, norming, the whole thing.
And so it's going to be real messy.
But I might mention that every time NAEP has new items, they in fact do something such as Eunice was describing.
They definitely look at how long it takes students to actually complete the entire item. And they do make some judgments about how much time to allocate based on student performance.
And you will have time to do that.
But we can't. I would be uncomfortable with us making that determination without actually having real time, students in real time.
But the people from states who have strong state standards right now should take maybe some comfort in knowing that there are people in the Department who are currently mapping the NAEP, sort of the goals in NAEP, reading and mathematics to state standards to double check the match there so that we don't have something that is dividing people's attention and their resources.
Actually, I was pleased to hear it because it's something that I have suggested for a long time should be done to make sure that this is aligned with at least a portion of what most states have in their standards.
I would rather for us to make the decision, even with some, you know, freedom. But let's make it clear what we want to have happen.
And I think it not only affects the test in general, but also those accommodations.
I think when we get to the part of the specifications document, I really do think that's going to be very, very crucial. And we need to have that discussion.
However, some people have only had a chance to glance at it.
And I'm wondering, and I just need to hear from you, whether or not you feel you would like to have a little time to look at it and then we can go back.
The main thing we wanted to do, the way we wanted to handle it is to have people key in, first of all, on the key issues.
What are some of the major points that you're concerned about in any way whatsoever?
And then, we eventually will be going through it almost line by line, if not line by line.
David is shuddering at the thought.
(Laughter)
(Laughter)
There may be one, two, a few items that we've got to come back and really look at.
I really think this is already taking shape. But I really think that after this, we would hope we would really be moving forward.
Sharon expressed an opinion about read time.
Anybody else?
Okay. Gloria said so, too.
Then, why don't we do this, Sharon, what would you say would be a reasonable amount of time?
(Laughter)
Did you just come in this morning?
One was on page -- and then just directly to tab F of the summary. On the second page, there are three bullets in the middle of the page. And this is more about style and tone than about necessary decisions.
But no bullet talked about the ways in which the test could -- it says, "It could be used to develop a common ground for dialogue among educators and parents about what's important in reading."
And there is a point that isn't exactly captured in the document, but could be or should be, if we still agree on that point. So that was one.
On page 3 under day two, the third bullet talks about the kinds of directions that students should receive and especially the notion about encouraging them to complete all items.
That set of ideas isn't exactly in the document. As I said, the natural place for it to be is under the test administration test section.
There is a list of things for administrators to do. And in fact, if you look -- go back to page 33 of the draft, it says, "include text for introducing the test to students and preparing them for testing sessions and to see if this is the kind of things they may not exactly be" -- I don't know if tips is the right word.
But it seems like a natural place where those kinds of instructions ought to be part of the overall document. So that's the second item.
Third is on page 4, the bullet says, "ECR items will not be used in the intertextual portion of the test" which was a pretty straightforward decision that the committee made at the last meeting.
As I remember, Steve led us through a variety of options for being more precise about the test specifications.
And somehow, we left that out. But it's easy enough to put it in.
Longer items, the last item is on page 6.
Tell me if I got this right, Eunice.
And that is that it says here in the middle of page 6 in the second quote under the distribution of items, cross cognitive reading behaviors and passages, " If there are only four questions for this kind of passage, that is -- yes, for this a short passage, then either a personal response or critical stance item may be selected. This rule of thumb need not apply to the intertextual component as this task naturally engages readers in interpretation critical analyses."
Was that the last one we sort of overlooked? Am I right about that?
(Pause)
Now, we will move on to major issues, concerns.
I don't believe that's correct. It's not going to provide that type of information for an individualized diagnostic assessment.
It could serve as an indicator when additional assessment is warranted.
You almost need a "more" in there to indicate that more testing would be needed to obtain a profile of a reader.
(Pause)
Then, it goes on about student performance in reading compared to national standard.
I guess I'm still not sure what instructional information is provided. The first thing says, when we look at the overview, we're going to tell parents and teachers how their students -- where they are basically on their development.
Now, we're implying we're giving instructionally useful information. And I don't think we are.
Jack and then Ginny.
It seems to me that that's the theme that needs to be consistent through the entire document.
And I think the kind of statement that Audrey picked up on 37 is not consistent with that, saying, we're going to tell how now you should shape your instruction.
I don't think the test is going to be designed to do that.
Students there are given (Inaudible). And here, the primary goal is that it returns information to teachers about their students' performance.
It's put the shift in another direction. I think we need to be consistent all the way through on how the scores are going to be used and by whom, who was this test intended to inform.
I would argue that we have because the framework is rather rigorous in the sense of what it is accomplishing. Okay.
So therefore, I would take the position that we could have an impact on instruction because we're making a general statement of what we think the outcome would be -- should be of instruction.
And we're creating a test that demands certain kinds of behaviors, performances by kids to do it.
Then, I would add to it my class scores -- am I getting a measure of how well my class is doing towards meeting that global outcome?
I don't see it all as diagnostic. But I see it as a statement of what it is that instruction should be aiming at.
So I'm not as critical of that paragraph as you seem to be, but I think it has to be, where is the information being --
And I don't think it's doing that. What you communicated is that there is a framework and that if a teacher's class has difficulty with this, then maybe the teacher needs to consider the dimensions of that framework.
But that's a pretty complex construct that I don't think is reflected in this particular kind of statement.
The other thing -- I think this notion of purpose is secondly important to this whole test.
The other thing I'm very concerned about is the issue of aggregation of any of this data. This issue has come up before.
And I think we have to address it squarely, whether in fact we're saying these data should not be aggregated.
And that begins to touch on it because now I'm saying here's how the class performs.
I'm not saying it's addressing all the dimensions of the aggregation issue, but it begins to get us in there.
And I think it's one that we have to discuss this morning and have some kind of conclusion about.
I mean, that has been communicated to me by people outside the committee and actually some people on the committee.
For example, if I'm going to give this assessment, then I'm going to look for the following school year and plan accordingly.
I provide more opportunities for children to engage in informational text if that was an area that I felt I had not done an adequate job in.
I might look at the four stances and do more around the area of demonstrating a critical stance.
So I think it's important that we keep in mind, any time any assessment is given in a classroom, it will to some degree impact classroom instruction.
Because any time that we talk about assessment, we talk about all the assessment activities that occur and how all those pieces of information contribute to form a picture.
And so, although no one piece is going to drive anything in particular, each piece is important.
So I worry that we make statements that imply that you're going to be able to do that.
I think we have to offer caution to the fact that there isn't the degree of technical quality that would allow that kind of interpretation.
And my understanding was that that's why we were going to have the supplementary materials to show what the implications for instruction might be.
I think most of us sitting around this table could very comfortably look at the stances and mold that and put that into instruction.
But I do know, having worked with a variety of teachers, that many times that just by looking at an assessment, they can't then change it and mold it into what they see as instruction.
So I think at least I agree. I think we have to be clear to say that there are some things that you can understand from this assessment, but there are also going to be some things that we don't get now.
And that's why there is a need to have other types of assessments that are going on in the classroom.
Or do you feel comfortable with the sentence that the primary goal in fact it says is true?
I mean, you don't have to do that right now.
But I'm wondering, do you feel comfortable with that? Or you would prefer that we come to some adjustment of that sentence?
It's going to give us some information about what the student is doing.
But I think that teachers are going to have to look at it a little more carefully before they decide how that's going to impact their instruction.
Moddy and then Ginny.
Does it mean individually useful? Or maybe, we should the use word "curriculum."
Because I agree with Alice and what Marsh‡ was thinking and with what Marilyn started with that that has kind of been the point of this that we do want to kind of model instructional in sort of a general way, maybe in a curricula way. We want to provide that kind of information.
But it's not individually directed instruction that it's going to change.
So maybe we can get the wording around to that. But, you know, it's the bigger picture that we're looking at.
That seems to be something that's out there.
What information exactly will teachers get because there obviously are aspects of reading that this test is not going to address?
So I think we have to be very clear about what it will address in 90 minutes’ time.
If you look in the overview which is the first page of text, the first paragraph under the overview about half down, it says, "The test will not provide prescriptive diagnostic information." And then, "However, it will provide an indicator."
In the spirit of that, I think you ought to make this sentence align or this paragraph align with that statement and then elaborate on it.
And, you know, I think the direction is clear.
And this, the overview pieces are taken out of the policy kinds of statements that were originally made by Gary and that I've heard from Mike Smith and so forth.
So I think what you do is -- the point of this paragraph is to talk about the need for studies, the paragraph under discussion.
So what we need to do is just tweak the paragraph, make sure that we don't make statements that are out of alignment with what's in the early part of this.
I think we just modify it to be consistent with the introduction and move on.
Elaborate the last sentence of that first paragraph. It says, "holds the potential to provoke positive changes across the American educational landscape."
By elaborating, extending, or standardizing curriculum or something --
(Laughter)
(Laughter)
And that is, I understand the point about this is only one assessment amongst many and that by itself it can only get you so far.
And, Audrey made the point well. And I think we can sort of attend to that; that it's not going to be a diagnostic instrument is also I think clearly understood.
What's a little fuzzy is -- at least for me is this notion about whether it's going to provide anything useful to teachers.
In other words, forget about aggregation.
This individual score and the individual results and all the variety of individual information, is that just for parents and students?
Or could teachers find any utility at all, as I've heard Alice suggested, like in informing their overall practice, whether it's informing their general plans, it's informing the way they address individual students?
Or are people around here suggesting that this is such a modest instrument that it can just be ignored and it's not attended to at all?
I mean, because that's where the conversation -- part of the conversation is going. It's sort of like, well, it ain't perfect. And so don't pay too much attention to it. And just do other things.
And, it's something to give to parents and kids. And you cannot think about it at all in how you go about your business.
And we really -- and David's question is hanging out there. So I will make mine quickly.
And that is I want to emphasize what Ginny said. And I believe in the overview, there needs to be a statement added about the primary consumers of this test.
And that those primary consumers are, number one, students, parents, and teachers. And I think that they are missing.
I originally wrote down parents and students under the bullets somewhere at the very bottom of the overview page.
But I believe there needs to be more emphasis on that.
And I'm not going to wordsmith how it should be included.
Gloria, do you want us to strengthen the force of that second sentence, as well, with respect to the privacy of that group of consumers?
Every time we list a string of people, they should be listed first.
I don't use the term "parents" anymore, unless I say parents and families. Or typically, I'll just say families, although there are people who say parents and families. So, however, I'm --
And I think we should maintain that consistency. And Ginny raised it. And Audrey even raised it when we talk about who's involved. That's all.
Diagnostic tests are very, very different.
And I know because we know this from many tests in the past that tests that were meant only to provide aggregate scores were in fact very often used for all good reasons with intentions by districts and teachers to try to make diagnosis implications, inferences from them.
And they're not necessarily the best way to go. And I'm afraid that we need to be very careful.
It seems to me there are two things that are happening. And I'm going to agree with Moddy in one aspect.
It seems to me that the framework and the items do in fact serve as models to suggest features to be included in the K-4 curriculum that are supportive of high level reading comprehension, something of that sort. That's number one.
So that's one-half, one-half of what I would talk about.
The other part would be that this instrument may provide information to families, students, and teachers regarding possible follow-ups and/or further in-depth analysis.
So that it then requires discussion and further steps before anything else is done.
But a student does quite well or quite poorly not for the reasons that first appear when you're looking at an individual student rather very large test selection of students.
And then, the second part is that it does point to directions of possible follow-up or it points to -- it may imply the need for possible follow-up.
Later on, I think it would be very important for there to be enough models so that in fact school districts and curriculum developers and so on could think very seriously about what could be offered.
And I know that Mike Smith is obviously talking about this a lot, how can we change things, the K-4 instruction?
Sharon is going to talk now.
This business if we put something in there about curriculum, Sharon, I want to hear your thoughts on specifically saying K-4 because I think there are people who worry that people are going to think this is a test on fourth grade, period.
And is it -- in the overview section, could you even have a bold face heading that might say what this test does and what this test does not do, but worded better than that?
But would it make it that clear? Or is that not appropriate for this kind of framework?
I mean, to me, if somebody, a parent or a naive reader picked it, okay, what is it going to do? And then, what is not going to do?
Barbara.
And one of the things in my mind that we need to separate out here is what will the actual reporting of the test results consist of and do and not do?
And then, what will the supplementary materials contribute to people's ability to use the test results and interpret them and then inform what they're doing in their classrooms?
And I think in my mind, that is an important difference. It's one thing to get a score and say my kid or kids or students or whatever do or don't match these standards by about this much.
That is not highly informative in terms of instruction. It's a gross cut.
But all the other stuff, when I see what kids are being asked to do and the kinds of answers that they are making, and I say, well, my kids probably answered it like this.
And that's where I think the insights into instruction will lie. And I think to pull that, to tease that apart a little bit more clearly would help the document because as we were talking, I was looking back under the first paragraph under supplementary materials.
We talk about professional development and stuff, but we don't really say in that first paragraph that we will have supplementary materials that will help, inform teacher's instructional practice, help them think about what they're doing literally in their classroom.
I mean, we're implying it when we talk about professional development, but I think we need to say it right up-front.
And the other thing is that it's the very last thing. We say in fifth they'll have a list of curricula resources.
Well, first of all, they better have more than just a list of curricula resources.
I think that the examples and the released samples of kids' work and so forth, that's going to be the powerful stuff.
But that stuff is not just powerful as examples of assessment. And we've talked about assessment system and models of student work for professional development.
That stuff informs instruction. And the first thing that we ought to say in that paragraph is that you're going to get some resources connected with this that will help you take a look at instruction.
And Marilyn just wrote me a note saying the other piece is to not only inform instruction, but probably almost more importantly in our days of talking about a wider partnership in education and that is to inform parents of what is reading today. What does reading consist of?
They are not often accustomed to thinking about what they do as readers and then thinking of the implications then for their kids.
Rather they often just substantiate what they may have done several years ago and rightly or wrongly and in first, second, third, fourth grade.
So I think that that's going to be an important aspect, too, is to give them those models, too.
But that's not the test itself. That's in the supplementary materials, I think.
It's implied in the kinds of descriptions that we make.
But I'm kind of sick of hearing people say 44 percent of the kids can't read a single word because they fell below the basic level on NAEP.
And if we're adopting NAEP standards, we are setting high standards for kids.
But I don't think that comes through clearly enough. And I think that is one of the real strengths of what we're doing.
But then, when we talk about the development of item and test specs, we say it's going to be informed by the framework and it's going to be --.
I mean, let's be straightforward and say that it's based on the NAEP framework.
The other issue is in here, in the third paragraph in the second line, we say that this process is going to be part of an effort to move assessment forward.
We're not moving assessment forward, are we?
We're not moving the assessment forward. Because of some of the constraints we're dealing with, in some ways we're moving it back. And in some ways, we're holding it steady.
But we're definitely not moving it forward. So I think we need to take that out.
You might be moving some other stuff forward.
I mean, I think that one of the things you're doing, the real thing is to increase achievement.
And that maybe doesn't go in that paragraph. That's what we talked about.
And I think we need to get that in more than one place. We need to get that at the beginning.
We need to get that when we talk about scoring so that people fully understand what we're talking about.
I support that 100 percent coming from my world of assessment.
On the other hand, I think where we have the opportunity to really make a difference is in terms of the kind of communication about test results and test frameworks.
And by virtue of going to parents, by virtue of going out there, by virtue of the list of publications that you came up with at the end of the meeting, I think this is where you will be able to say that this committee will have an impact on moving assessment into a different sphere in terms of what it communicates.
And I think it came out of the list of -- Marsh‡, you weren't there to listen to the list of --
It could be an impact in what instructional outcomes could look like.
It could be an impact for parents and kids in terms of what they're expected to be doing, okay, how they should be expected to be performing.
It's not just the score. It's not just the test, but it is the whole package.
For some states, it's the first large-scale piece of assessment that includes the opportunity for open-ended responses.
So it's the first test to validate on a national scale the use of more contemporary forms -- what we used to call alternative assessment which is becoming more mainstreamed.
This is the first test that is going to come with a set of supplementary materials that expands the notion of what a large-scale assessment is and that tries to situate it in a context and help schools to situate it in a larger context of assessment.
And so for all of those reasons, I think it is. And the ways we're thinking of reporting, I think it is what you said it wasn't.
(Laughter)
When you look at what's happened in California and Texas, pay very close attention to California's actions following their NAEP, the release of their NAEP data on their reading test.
And we just can't underestimate the kind of impact that is going to have.
Marsh‡, you say it keeps us at a standstill. In Texas, this could move us way forward.
It probably depends on which state you're in and how progressive you are.
I come from a very conservative state where changing assessment, from reading a paragraph to multiple choice questions to a whole passage was a major event.
So it probably depends on your perspective as to where you are.
Granted, you are in a place where you're doing terrifically progressive things, but --
The conversations that I have, the struggles that they are having with this test right off the bat was the percentage of multiple choice.
And so knowing that we have to deal with that and knowing that, yes, there are states out there who are still thinking 100 percent multiple choice is hunky-dory.
But to have a statement that we're moving is almost like a slap in the face to those who are way down the road and still want to participate.
So that's one thing. The language, I can deal with it if the language is somehow clear enough that we're not just talking about the test itself, but how the test, the processes that may be associated with that, instructional materials, anything.
But just that statement, you know, just doesn't get us there.
It seems to me that it's very important for a different constituency to be able to say that these in fact are well-founded instruments we're basing this assessment on, well-founded instruments.
That it is not something that either is used for experimental or advancing at least within the state of assessment. And it's not.
So I think it's important to say what we are doing is offering a much more widespread use to use assessment to make a very substantial difference in what students learn in the early grades or are exposed to in the early grades.
We are committing districts and states to be able to re-think their own curriculum if they would like to do that.
But I think it's important for the states that are moving beyond this. And those that are nowhere near it and have a very conservative constituency be able to say this isn't something that at this point could be (Inaudible) in fact we're following the framework that has been in place for quite some years at a national level.
Ginny and Audrey.
It's almost time for a break. So let's finish up with the two of you.
And then, David, if you are just about --
I think I'm agreeing with what Judith is saying that we just need to leave that kind of language out and let states make that determination for themselves as to whether we're moving it forward or not.
(Laughter)
I just want to make sure the message that we're communicating really speaks to the accuracy of what we are doing.
And that's an important accomplishment to be recognized and not to be undersold and not say no one has ever done this before and open yourself to attack.
That is why it is an experimental thing.
But to understand that this is a next natural progression after many years of work at the local and the state level.
And so you give people credit where they've done important work and say we're making this available to everyone and build on that.
And I will list those, what I think they were when we begin. And you can let me know if there are other things that you think need to be added or if I got it wrong.
Fifteen minutes.
It seems to me that we really spend a good deal of time on some suggestions for the overview.
And so I jotted down some of the things that I had. And I would like for others to chime in if there are some points of difference.
I just want to say though that we have to be careful. The overview shouldn't be overloaded. It should have important elements in it that we think we want to convey immediately, but we have to be careful not to overload it.
We want to express clarity about what the test will do and what it won't do, and how the supplementary materials will contribute and make a statement somewhere within the three existing diamonds, I guess at the third diamond something about the supplementary materials for parents and students.
And, Gloria, you had raised that issue.
We want to make it clear that the assessment will have curriculum implications for K through 4. And that can certainly be put in, embedded within maybe that first paragraph.
Stress the use of high standards. And that these youngsters will be assessed against high standards.
Stress the fact that it capitalizes on the best thinking and research to date, that it builds upon previous work and makes that available to everyone.
It has a firm foundation, something, words to that effect.
And either add families to parents or just include families where we talk about providing information for students, parents, and teachers.
It could be parents and families or just families.
Now, those are the things that I had. Is there anything else?
Eunice.
Marsh‡.
I thought we said we were not going to do that.
(Pause)
Marsh‡.
(Pause)
And the first thing we talk about is their positive attitudes and positive self-perceptions.
I know that -- I have two concerns about that: one, whether we should have that statement in there at all.
And secondly, if we put it in there, that it should come later after we describe the attributes of a proficient reader.
Why I'm concerned about putting it in there at all because although it's true that that's the characteristics of a proficient reader, when people read things like that, the first thing that they come up with -- this is one of those flags from a policymaker's perspective.
People think, oh, they're worried about kids feeling good about their reading. They can't read.
Skip all the other parts that say proficient readers feel good, you know, etcetera and so forth.
We're not assessing that. We're simply talking about one of the attributes.
If it's not important to put in there, I would say take it out because I think it's going to cause more problems from those who read things like that in ways that we don't intend than the good that it would cause in terms of just giving people a general description of what readers are about.
Alice, I saw you shaking your head. What do you think?
(Laughter)
But if we're saying that -- I mean, this is what I deal with when I talk with parents all the time.
There are many children who can read and are able to read, but they choose not to read. And they are not growing in their reading.
They are not -- I mean, I just think it's important for us to say -- to keep it under the characteristics.
These are characteristics that do separate accomplished readers. I think a lot of them -- NAEP data show that people who read on their own time tend to score better.
And that's because they have a positive attitude. And they're making their choices about what they're reading.
So I would like to have it somewhere, but I don't think I agree with Marsh‡ to put it right up front there in the first paragraph because it could cause some people from reading on into it.
One of the attributes of people who do well in math, they have positive attitudes about math and they do math things.
I mean -- and I guess if I were -- well, that's nice.
Again, if we put the students primary, there first, and then policymakers are some place on the list, students need to know that.
It's a qualification of a good reader. Or a part of being a good reader is your attitude about reading and your uses for it.
I agree with Marsh‡ that probably it shouldn't be first out of the box, but I think it needs to be included again because of the audience that we're trying to appeal to. And that's the student, not the policymakers.
So I don't think that we really do need to keep the sentence.
I mean, they choose to read a variety of materials, recognizing that they read often to develop their own criteria. They function successfully. And they take personal satisfactions in their reading.
I think those sentences say what we've been trying to say of why we think that that's important.
But it certainly has implications for students, for families, for teachers.
And even though we're not assessing it, students who read and choose to read are going to be successful readers.
So couching it in the attributes of what a successful reader is in terms of being strategic and motivated and engaged are really important.
I think you can strike a compromise by moving a sentence to the end of that paragraph that says in general, accomplished readers have positive attitudes about reading. Stop.
None of this positive self-perception stuff.
Then, what I would also do is take the number A under that list that really expands this and move it down to the last on the list instead of the first.
Not only does it take care of your concerns, but the other thing that it does is it puts first in that list something that is near and dear to a strong contingency.
And that is the fluency statement. And I think really when some interesting people come out to question our endeavors in this area and want to know why we're not testing for anemic awareness in the fourth grade, that statement B being first on the list of characteristics of good readers is going to be a very important statement to be able -- not to have buried in the list.
So that deals with people's needs to say something about positive attitudes. It doesn't get quite into so touchy an area as positive self-perceptions which I think is much more the red flag on this piece.
And then, the second step would be to move A down to the end of the --
Eunice, you wanted to make a point.
And also, Barb, so you have, in general, accomplished readers have positive attitudes about reading.
Do you want to say "and about themselves as readers"? Or do you want -- no. Okay.
It's just that that phrase "positive self-perception" is a red flag to some contingencies.
(deleted)
(Sample 2)
I have a couple of things to talk about. Things are moving quickly with the Legislature, as you know. Over the holidays -- we have what is called a Faculty Legislative Liaison Committee, a rather lengthy title for basically our radical faculty group t hat's trying to figure out how to work with the Legislature. And this group worked diligently over the holidays. We met with the Governor, in cooperation with NC State. We had representatives from both NC State and Carolina to meet with the Governor to push two primary goals that we've been working on for a couple of years. First is competitive faculty salaries. We continue to work on that, to get us back to where we were in the early 80's. The second piece that we've now begun to discuss is greater support for graduate education. This is a tougher sell. We haven't really talked about this in a way that the State understands yet. And so we're working hard to make that understandable and have the State understand all that that brings to the State. We have made an economic argument to the Governor which he endorses and supports. A couple of years ago Michael Luger in City and Regional Planning did an economic analysis, and he's updated it. And basically what we know is that for every dollar the State gives to support the University at Chapel Hill, we generate another three to four dollars for the State. And so basically every year we are generating almost $1 billion for the State economy. So if we need to be speaking in economic terms, which I think we do need to be speaking about at this point, we would say that we are a great investment for the State. And so when the State is in a situation as we are in now, in economic good times, we would argue that it is time to continue investing in the University.
However, we are now at a point where we should probably be very pragmatic about the political environment we're in as well. This morning we saw that the UNC Board of Governors has begun to talk about which programs we're going to cut. The measure they 're using to decide which programs they would cut is the number of graduates. I think most of us would argue that may not be, certainly wouldn't be, the sole measure we would want to use to decide which programs to cut, that there are a number of explanations for why we may have small, very high quality programs. And graduates take longer to graduate in some programs than others. And part of it may be, especially at the graduate level, that they're not graduating because we don't give them enough to live on. And so they have to be having two or three jobs to support themselves while they try to get through the degree program. So these are complicated issues. And what I would argue at this point is that we as a faculty, rather than criticizing the Legislature or the Board of Governors for the kinds of solutions they're coming up with, is what we need to say is that we want to be involved in this process. If cuts are to be made, we need to be in that conversation. We need to be involved in deciding what measures should be used, to decide what should be cut if we have to cut.
So, what we've been thinking about, that is the Executive Committee of the Faculty Council, is to -- this is a bit of an awkward segue, but I think it's related -- is that what we've been working on is, there is a convergence of planning efforts right now. The Chancellor just spoke about the land-use planning. You've been seeing that all over the newspapers. And a number of faculty are involved. And a committee that's working diligently, with Tom Clegg as its chair. That's one planning effort. At this point we're beginning to say also that -- I'm sorry -- so the planning effort that's also in place or moving is what we have coming out of the SACS reaccreditation process. So for a year-and-a-half we've been involved in a self-study. We've basically been self critical, looking at what are we doing well, what still needs to be done, what's missing. I see this as a possibility right now, that we put these pieces together and we start looking at the future in a way that, so we will be prepared to speak to the Legislature about where we want to be going in the future, where we might, could perhaps cut if we need to. Like that. So we will be talking more about that planning process, getting that in place in a way that really will work for us. I encourage you all to speak with us about how this can proceed. Some of you are experts in planning. Some of you have expertise in thinking about the future in a way that I perhaps don't. I've only just come to this, thinking of 25, 100 years hence. Some of you do that every day. So, if you do, please let me know about that. And we'll start talking about how we can make this happen. And if you want to talk about it right now, I'll be happy to. I have a couple of other -- So that's where we are. That's what I've been spending a lot of our time thinking about and the Executive Committee's been thinking about -- besides basketball tickets. That will come up again.
I'll tell you about it since it's been in the press again. We will talk about it at the next meeting probably. It comes back to the Agenda Committee in a couple of weeks, and so the Agenda Committee will decide whether we are going to talk about it here. I've heard some comments from you. If you want to give me more input now, fine. As you've read in the paper, I think this isn't the most important thing we could be talking about right now, but John Swofford assures me that this is an issue that never goes away. So perhaps we just have to keep dealing with it. So, your advice and counsel on that are appreciated well.
Three announcements, or I would say, in the church I go to we call them "invitations," when they're announcements -- opportunities for you as faculty. George Jackson is here. George Jackson is the Academic Affairs Officer for Student Government. I think I just botched his formal title, but we've been working on a couple of issues together and he wants to speak to us about the Carolina Course Review quickly.
And thirdly, there is the eighth annual Show of Hands for Peace and Unity. How many, anybody ever participated in that before? It's on Wednesday, January 18th. It's in support, it's in celebration of the Martin Luther King, Jr. birthday, and apparently it's a very moving experience. It's been primarily students previously, and we'd like to have some faculty participation there as well. It takes place noon to 1:00 this coming Wednesday, on Polk Place. Where is Polk Place? Is that the Old Well, or is that behind the South Building? So, great, I hope we see you there. Anything else? Invitations, announcements from you. Opportunities, comments, criticisms, celebrations?
(deleted)
(Sample 3)
I would like to welcome everybody on the part of the Department of Education, the Council of Chief State School Officers, and members of the committee.
And I think it might be good to just go around the table and introduce ourselves, since there are some members of our committee here for the first time.
And maybe, we'll just start in the corner and have everyone introduce themselves at the table.
(Whereupon, the introductions took place.)
Gary.
Well, I also want to welcome you here and welcome our guests. This is a very important meeting for us.
It's the first public hearing of the Math Committee. I think there will be one other public hearing as well later on in the process. And there will be two as well for the reading.
We've had about five or six public meetings on the test overall.
But these are very important meetings for us. They give us an opportunity to hear from you and to think through what you say. And what you do say does affect policy.
We will listen carefully to what you have to say, both the Math Committee and the Department of Education on a broader level.
And what we've done in the past, the transcripts of the meetings and the summaries and the minutes, we take those back.
And we go over them. We think it through. We see how what we're doing is consistent with what you would like us to do.
And, you know, in general, I think we have a real good policy here, this National Test. I think it is a good idea. And it's the right time.
But we want to have you to buy into it, to understand it, to support it. And we will make whatever modifications that we need to, you know, satisfy you as best as possible.
So these are very important meetings for us. And we appreciate your coming here to give us your comments.
Just in general, I just want you to know that this is not -- this whole activity is really not just to create a new test.
There are many good tests out there. Some not so good, but many very good tests out there.
This is really not the whole idea of this. It's not just to create a new test.
The idea of this or the purpose of it is to improve student learning. That's the whole thing.
And how does this work? Well, I think it will work by just the presence of this test, I believe, will energize the American educational system, just being there, not to mention the information that it will provide, but it will energize the American educational system.
And I think it will serve as a scaffold that will stimulate other activities that will surround it.
The whole idea here is to empower teachers and parents with information that they don't currently have.
When we give reports from the TIMSS results or from the NAEP results which are national surveys, policy makers use that information.
They consider it to be important. They use it for policy purposes. They make decisions.
Not a single teacher, not a single parent or student has that information for themselves.
No teacher knows what their students do on the NAEP test, for example, no parent, no student.
The idea here is to empower parents and teachers with information they don't currently have.
This test also really is the next logical step in the standards reform movement, the National Content Standards Reform Movement. NCTM standards have been around now for almost 10 years.
And other associations have developed standards in reading and other areas.
Those have been considered to be successful. But again, a lot of the work on the standards movement are really at the policy level, at the national, the state, maybe the district level.
There hasn't been a lot of penetration down into the classroom. This test, part of what this test will do, I think, will take national standards and move them down into the classroom.
So for the first time, parents and students and teachers will know how they stack up against national standards and international standards as well.
Another aspect of this whole activity, again from a policy point of view, is that we want to set -- the whole thrust of this is to set the same high expectations for all students.
We don't want to have just the good curriculum in the suburbs, curriculum with lower expectations in the inner cities.
We want to have the same high expectations for all students. I think this will help to bring that about.
It won't guarantee it. It will help bring it about because the same students in the inner cities and the suburbs will be taking this test.
So this is really a different kind of test. It is another test, but there are things about this test really that are very different from what I think you might be used to from other tests that you observe in local and state testing programs.
One is, of course, the President's involvement. The President has committed himself throughout the remainder of this administration to talk about education and this test.
And you've seen that in the past. He's had many meetings on this. He's going to have many meetings in the future.
So this is really the first time where we actually have the President of the United States out there talking about education and talking about the importance of this test and what it will do.
This is really the first time that this country has had a test in which it will be released to the press right after the first -- right after the administration of it.
It will go -- it will be released to the press, to the Web, along with scoring guides.
Parents and teachers will be able to see and students will be able to see what this test was all about.
This will be a lot like the test in Europe where when they are released, parents and the public talk about the test. The items are in the press. And people discuss it. It's a topic of conversation.
This test will be developed in a fish bowl. Most tests are not developed that way.
Every meeting that we have on this test, every single meeting will be a public meeting. There will be transcripts of those meetings.
And the only meetings that we will have that will not be in public will be those in which we work on the items themselves.
But the item and test specifications will be a public document. We will have a sample of the test available prior to the administration in 1998 so that the public can see what the test is all about.
It's all being done in the open with lots of opportunities for stake holders and other constituents to have input into the process.
We want that input. And it's being designed explicitly to get that input.
This will be also I think one of the first tests that will be developed in such a way that we are going to make the test understandable.
The metric and the reporting is going to be focused on making the test understandable to parents and teachers.
Most testing programs, even the most well intentioned, don't put the energy and the effort into trying to make the scores and the information understandable and useful.
That's a primary goal of the testing program. When parents look at this and teachers look at it, they will say, yes, I understand that. This is obvious. This is a good thing to do.
A lot of effort will go into that in the development contract with focus groups, with students, parents, and teachers.
This is also a test and possibly one of the few tests or the only test in which students will get information. We will get information on individual students about how they stack up against other students in the nation on nationally developed standards, developed through a national consensus process.
And we will do that through the linking with NAEP. And so what we will know, we will have information on each student about whether or not they are below basic, basic, proficient, or advanced.
And that information simply is not available in any of the testing programs.
We will also have international information. Students can see how they stack up against students in 41 other countries.
So the whole idea here is to energize the educational system, make this test be -- have a catalytic influence.
And I think we're really entering into a new era here. In many ways, this is historic. This will be information that the educational system simply hasn't had in the past.
And so you are an important part of this. I'm glad you're here. I'm glad we're having this meeting. I'm looking forward to it.
Thank you.
And we do have some people who have already indicated that they are representing.
And we will begin this morning with testimony by Tim Schlenvogt whose representing the National Association of Secondary School Principals.
We welcome you, Tim.
As I mentioned, we are here to hear from individuals and the input that they have. We will allow you to present your testimony.
There may be some questions from members of the committee relative to specific remarks you've made following your remarks.
I would like you to begin by identifying yourself, you know, technically, the group that you represent.
And if you have written comments, I would appreciate having a copy at the close of your remarks so that we can have that to make sure that anything that is in the record is accurate and falls very closely.
So we welcome you. And I'll let you identify yourself.
I'm also a Pastor of NASSP's Middle Level Committee.
I want to thank you for soliciting our ideas and concerns, as you deliberate and develop the item and test specifications for the Voluntary National Eighth Grade Math Test.
My comments represent my views as a middle school principal and former mathematics teacher and the thoughts of Sue Galletti who is the Director of Middle Level Services for NASSP.
First, we believe that the instrument that is developed needs to be aligned with the curriculum that is taught.
"Breaking Ranks: Changing an American Institution" which was published by NASSP in 1996 makes the following recommendation.
"Assessment of student learning will align itself with the curriculum so that students' progress is measured by what's taught," echoes what Gary was speaking of earlier.
This call for alignment is further called for in the "Draft Standards for National Board Certification of the Middle Childhood and Early Adolescence Mathematics Standards" which was published in April of 1997. And that is found on page 37.
The previous quote I had from "Breaking Ranks" is found on page 11 in that article.
At the same time, accomplished mathematics teachers advocate changes in accountability measures so that such indicators become more closely aligned with instruction in important mathematical outcomes and therefore more accurately portray student learning.
Second, we believe that the instrument needs to encourage consistency of commitment to math reform recommendations.
NCTM has developed high standards. One of the findings of the recent TIMSS study of eighth graders was that where instruction mirrors a reform recommendation, students do well.
In fact, Japanese mathematics teaching more closely resembles the teaching envisioned by NCTM standards than does current U.S. teaching.
A further finding is that most U.S. math teachers report familiarity with reform recommendations, although only a few apply the key points to their classrooms.
NCTM practices have been validated by the TIMSS. We encourage that the Voluntary Test reflect the NCTM standards, include input from NCTM, and mirror reform recommendations.
We believe that assessment of students needs to encourage instruction and curriculum that reflect the vision of these standards.
Third, we believe that the instrument needs to emphasize high standards, encouraging our students to be competitive with students around the world.
Test items need to discover the degree to which students are able to discover concepts and principles underlying important mathematical topics.
They need to detect important relationships connecting content strands. And they need to use mathematical ideas and methods in significant application.
Test items need to encourage that all students understand algebraic techniques and procedures for transforming and simplifying algebraic representations, as well as understanding how to reason about relations and how to draw inferences in solving problems.
The test needs to encourage algebra for all students who exit the eighth grade.
The test should be problem based rather than multiple choice.
Fourth, we encourage that the Voluntary Test should not just be more testing. Too much time is already spent in classrooms currently preparing students for state standardized tests or district standardized testing at the expense of time spent on quality instruction.
Teachers should be implementing reform recommendations in their classrooms, teaching students what has been collectively identified as what students need to know and be able to do in the area of mathematics.
The test should measure the degree to which this has been accomplished.
It should not just be one more tool to collect data which is not aligned with curriculum and instruction.
We encourage that the Voluntary Test when developed to align with NCTM standards and instruction that responds to NCTM recommendations, become the state standardized test, replacing other state standardized math tests.
We support the idea that there be easy to understand reports that would provide students, parents, and teachers with a sense of what students know and are able to do against high standards.
We further encourage that end service be provided to schools on how to use the data to improve instruction.
Ideally, the assessment will be tied to end service that ensures that schools are accountable and that all students perform all standards -- or at standard.
States will consequently need to examine provision of resources that ensures that students are taught the standards that are expected.
Regarding details of the test, we encourage the test be administered on two consecutive days.
We support that calculators and manipulatives be used in keeping with the recommendations of NCTM.
Finally, while we recognize the political necessity of the Voluntary Test being voluntary, we would encourage that after a given period of time during which the test is piloted, monitored, and adjusted to ensure alignment with reform recommendations and best teaching practices, the test become mandatory.
If we indeed know what our students need to know and be able to do by the end of the eighth grade, and these standards are high, competitive, and agreed upon, we should expect that all students will be able to demonstrate that they have performed to standard.
It is only by requiring the same test, if constructed correctly, that we will ensure that all students be provided an equitable and complete access to quality math instruction.
Thank you for your consideration and your attention to all of the things that we brought forward.
Are there any questions?
And I think it also lines up well with some of the earlier deliberations that the committee has had.
So I think you will have a good result.
The one thing that I was curious about is early on, you said how important it was to align the test with the curriculum that is taught.
And you also made a statement about being consistent with the current reforms and with the NCTM standards.
And how do you think about that, or if it's on a chasm, at least that variation of that exists out there?
And how can this committee think well about satisfying those two very sound and legitimate issues that you put on the table?
I know that in our state, in the state of Colorado, one of the things that we are working on right now is meeting standards and benchmarks for curricula.
The curriculum that happens needs to lead to whatever the assessment is. If we decide what the assessment is, we have our standards and benchmarks. And we have our assessment over here.
Then, we figure out the things that happen to lead to that assessment.
Teachers can do that. Even if it is a range or a variety of ways, it can still get to that.
But then, teachers need to also be accountable for making sure that the things in all those little pet projects that happen internal in the classroom do indeed lead to some sort of authentic assessment.
And if the authentic assessment is this Mathematics Test and it is indeed an authentic assessment of what kids know, then the curriculum will lead to that. And teachers will be accountable for that curriculum.
I don't think our charge here is to devise a curriculum. The charge is to devise what the authentic assessment is.
Many different curriculums will still lead to the same authentic assessment.
And I think the variety or some of this disagreement from teacher to teacher as to what exactly is the best way to get a topic across to students to meet a particular assessment, that's okay, as long as it is leading to that assessment.
And that's a little bit different than the way curriculums have been developed in the past.
In the past, we've developed what the curriculum is. And then, we try to figure out what it is that we're testing, how we're going to test it, and how we determine whether it's been learned.
Well, this is a little different. And it is coming from our benchmarks and standards, whatever they happen to be.
And if you begin in a very concrete subject like mathematics, the natural evolution is that it will follow to other courses.
And I think -- I don't know if I've answered your question, but I think in that process that the variety of different curriculums that are there, as long as they are meeting the authentic assessment is allowable.
And that is the business of how to help classroom teachers think about using the results of the assessment for better instruction.
However, if there is the support of how to do the corrections, you know, the corrective action to make sure that kids are leading to that assessment or how to teach teachers how to meet and reflect on how -- what is the curriculum that is going to lead to that particular assessment as an end result, some sort of staff development, end service ideas that fit with that could certainly align with many of the other things that are currently being done to help teachers plan for leading to -- meeting standards and benchmarks in other subjects.
And I know that we internally in Colorado have worked long and hard on that over the last two or three years of how do we help teachers reflect on student achievement and either do corrective actions to make sure that students accomplish that?
And it can be done I believe, as an umbrella. Okay. Here are a menu of things that happen. Here's what you do. Here's a menu that you can check against.
And if that information is supplied to each individual building administrators or teachers, I think all of us have the ability to pick and choose things that will help us do that.
And a menu of things would certainly be an idea. I don't think it's the only idea.
One of the things that will come out of the work of this committee as it really rolls forward to whatever contractor the government would pick to enact the design and specifications that we're responsible for, will be a development of a set of sample assessment materials that instructors may use, both to test and as a base.
It could be used in the schools, a base for professional development.
Or it also can be used as a teacher, as examples to actually supplement their curriculum as students prepare for the National Mathematics Test so that they're quite aware of the different forms of assessment that appear on that actual instrument.
And such a book would be produced each year. So it would have student work so that teachers can actually see how rubrics are applied and see how students actually respond to questions and how that matches up with the objectives that those tests were written for.
Gary.
Again, as the test is being developed, there are many other initiatives in the Department, like the American Reach Challenge, the Math Initiative, and this joint group, working group.
The committee efforts there or some efforts there will be focused on the National Test.
And there will be resources of materials available as those groups work. And those will be connected to the National test in some way. We haven't gotten that worked out yet.
But -- so I just want you to know that that is going to happen. And we're looking forward to the report from the working group.
My question is, do you see some role, national role that we could play to -- for the end service and staff development, I mean, something that would be appropriate for the federal government to either support or think about or encourage, something like that?
As I suggested earlier, it may be the menu of items that you can do.
I think the position, first of all, establishing the idea that this is a commitment that we're making nationally and letting folks know that this is not just another thing that is happening.
And something that is a long, long reaching or a far reaching commitment by the government, by the Department of Education and that that also networks down into state departments of education that this won't go away. This is going to be here. And we're going to work at it. And we're committed to it.
And I think that would be score reporting. And that is maybe worthwhile to think about, the kinds of reports that will tell, be available so the classroom teachers can look at patterns of performance across content areas or other kinds of domain.
And that would be a different kind of report that one could get of individual student performance.
So the report might be helpful.
(No response.)
Thank you.
Do we have other groups that wish to address things and individuals?
(Pause)
I serve on the board of directors of the National Council of Supervisors of Mathematics. And I'm currently -- and I'm formerly a middle school mathematics teacher.
And I just wanted to address some of the information that I recently received regarding the National Test for Mathematics and a position of my involvement with NCSM as well as with our Statewide Systemic Initiative.
First of all, we definitely the alignment with the mathematics framework for the 1996 National Assessment of Educational Progress.
We support the wide distribution of items so that they are across the number sense, the 25 percent number sense, 15 percent to measurement, 25 percent in geometry, 15 percent in data analysis, statistics, and probability, and a 25 percent in algebra and functions. So that there is a balance of those items.
At the same time, we would like for the committee to think seriously about the big ideas of mathematics and not be tempted to address these various issues -- these various topics with a lot of isolated items.
So the big ideas of mathematics are being addressed.
Secondly, as far as the form of the test items are concerned, we would like to see as many items as possible addressing student's conceptual understanding of mathematics and their ability to solve problems in meaningful contexts.
This may involve some performance-based items, as well as constructed response.
We would like to see -- because of our work in Colorado with looking at assessments from multiple measures, we're comfortable with having items that are a balance of multiple choice, short answer, but also to emphasize the need to have some performance items as well.
As far as the question regarding calculators and the use of manipulatives, we would like to see unrestricted use of calculators and manipulatives for students, those that students could bring with them and those that they are most familiar and comfortable in using.
And we would like to have some items on the National Test that would require the use of calculators so that we can see -- the effective use I should say of calculators so that we can see whether students are in fact using the calculators appropriately.
Regarding your request for information on reports to parents on students in the community, we would like for those to highlight student's strengths as well as their weaknesses so that we have students who know.
Perhaps, we could use the format that we've been using with the National Assessment for Educational Progress that students are told if they are proficient, advanced, or partially proficient in each of the five content areas related to the ideas of mathematics.
The time framing for the test, we have been involved with the Connecticut State assessment program in looking at how can we provide time for middle school students and within a period of time.
So I would strongly suggest that you not consider one 90-minute session, ha