>> Welcome to our webinar, the IDC webinar.
The mission of IDC, as many of you know,
is to provide technical assistance
to build the capacity of states for collecting,
reporting, analyzing and using high quality IDEA data.
And you have joined us today for a webinar
on leveraging evaluation data:
leading data-informed discussions to guide SSIP decisionmaking.
I am Tamara Nimkoff, and I am joined
by my colleagues Kim Schroeder
and Debbie Shaver.
And just a quick little housekeeping,
as Sophia put into the chat,
everybody has been muted for the audio quality.
And for questions or to share comments,
which we encourage, please use the chat box.
And when you do so, just make sure
your little send to box says all participants.
So, in providing this webinar content today,
it's our intention that you will increase
your understanding of the value of data discussions for assessing progress
toward achieving intended outcomes,
and for informing decisionmaking,
that you'll increase the understanding of a structured process
that we're going to go through that groups
can use to guide next steps in SSIP implementation.
And specifically, a new protocol called IDC's Data Meeting Protocol,
and some related resources that are available to support you
in data analysis and use.
So, we'll do this in the next hour by first,
I'm going to set the stage a bit by reviewing the need
for states to be assessing progress
during this phase three, year two of the SSIP.
And the value of leveraging your SSIP data for decisionmaking.
Kim will then introduce you to the new tool,
the Data Meeting Protocol,
and talk about how it can be used to support data discussions.
Then she'll provide an overview of each step.
Debbie will then lead us in hearing
about a couple of example situations
in which the protocol has been use within states.
And she'll provide some information on resources
that we have available to support this process.
So, let's get started.
We know that the Office of Special Education Programs,
OSEP, has provided very clear guidance
on the expectations that states continue to assess progress of their SSIP
during this year two of phase three.
And that it's in so doing,
they assess both short-term and intermediate outcomes
to gauge progress
toward your state-identified measurable result,
or SiMR.
And perhaps most importantly,
using those evaluation data to make data-informed decisions
about the ongoing implementation of SSIP strategies and activities.
And all of this, for example,
is laid out very explicitly
in the SSP/ARP measurement table.
So, using your data to guide SSIP decisions
is really about leveraging data,
or, you know, taking advantage of the data
that you have collected in relation to the SSIP,
and using it as a tool.
The power of the tool is that it can inform your decisionmaking
about allocating resources,
about targeting areas for program and service improvement.
It also has the power to help build awareness and interest in skills,
both among your staff and stakeholders,
for that routine use of data.
Data-informed decisionmaking, you know,
really ensures that the data that has been collected
has value in your agency,
and among your staff, and then subsequently,
it really helps support improved quality of your data,
because you are actively examining
and using those data along the way.
So, now, I'm going to turn it over to Kim,
and Kim is going to kind of take over
and introduce this new tool
that can support you in actively using your data.
>> Great. Thanks, Tamara.
So, in thinking about the power of data-informed decisionmaking,
we have been working, over the past several months,
on creating this protocol to help guide discussions
that lead you to data-informed decisionmaking.
So, why a data meeting protocol?
We really wanted to provide groups
with a concrete way to move through the process
of talking about data in an organized, systematic way.
And so the protocol gives you a simple structure
to guide the conversation of a group around evaluation data
or other data in a meeting.
And we want everyone to be on the same roadmap,
and so we think that this protocol will help do that for groups
as they move through the process of looking at
and talking about data.
We also want you to move from just looking at data and evaluation results
to really making meaning of the results together.
And we hope that the protocol will do that as well.
And then finally, as Tamara already mentioned,
it's a way to leverage your data,
support the analysis and the use of all the data
that you are collecting to inform your projects,
processes, policies, your continuous improvement.
So, who is this protocol for?
We created this protocol to be very flexible,
so it can be used by different groups.
So, really, anyone who's engaged in making decisions
for improvement efforts such as the SSIP,
but other improvement efforts as well,
this would be an appropriate tool to use as you're working with groups.
It could be used with a single group,
so you might use it with a group of local staff
that are involved in a specific SSIP initiative
or other improvement effort,
or it might be a cross-section of different groups.
So you might have, as you see listed on the slide,
some state staff, some local staff,
and other partners and stakeholders
involved in a particular effort or initiative.
Again, it's designed to be flexible,
and to be used with anyone who might be engaged
in these decisionmaking processes.
We do identify a couple of key roles
that we suggest you assign before a meeting begins,
and it really helps frame the protocol.
We suggest assigning a protocol lead,
or possibly leads, could be more than one individual.
And these people have responsibilities
both before and after the meeting.
A lot of organizational and communication
tasks in getting the meeting off on the right foot.
A facilitator to guide participants through the group discussion process,
and we want to the thoughtful about,
is this an internal staff member,
or would it be more appropriate for your group
to seek an outside facilitator such as a TA provider
to facilitate this particular discussion?
And then the other roles we suggest as key roles
are a note-taker and a timekeeper.
So, how might the protocol be used?
Again, the key for us was to make a flexible tool that has many uses,
and we really mean for the tool
to be re-used with different groups and different data sets
as you work with different constituencies in your day to day work.
It could be used for a single meeting with a group that may only convene once,
or in a series of meetings that are part of a recurring decisionmaking process.
So, a group that meets frequently could use it as a guide
and to structure their meetings.
And in terms of what can be addressed through the protocol, again,
flexibility can be used to facilitate discussions
about a program's processes, implementation,
the extent to which a program achieves its expected outcomes.
It's meant to be used with different groups,
different data sets, multiple times.
So, let's jump into what the protocol actually looks like.
We've divided the process up into three parts,
so we discuss things to do before the meeting,
during the meeting, and then after the meeting.
The before the meeting protocol steps really address planning
and preparing for the meeting,
and identify things that a protocol lead or leads
should do in advance of the group coming together.
During the meeting is obviously the heart of the process,
and is really about the facilitator guiding the group
or the participants through the discussion,
based on the data that has been identified as critical to the objective.
And then, finally, after the meeting,
protocol leads will provide a recap, next steps,
and communicate those to the participants.
And so we want to give you a little bit more in-depth overview of the steps
as they're laid out in the protocol itself.
So, before the meeting, we have really five key steps.
The first is to determine the objective.
To think about what you're trying to accomplish.
You're bringing this group together. What does the group want to know,
and what evaluation questions are you trying to answer as a group?
The next step would be, then, to identify the data.
What's going to be helpful in answering those questions
or reaching that objective that you've already identified?
Next is to identify participants and key responsibilities.
We just a few moments ago talk about key responsibilities
in terms of a lead, a facilitator, a note-taker and a timekeeper.
But then in this step we also encourage you to identify participants.
Whose perspectives do you want represented?
Who do you want around the table
as you discuss the data to reach your objective?
Before the meeting, we also outlined organizing the data.
And this is collecting the data,
but also thinking about how best to present it to the group.
Does it need to be aggregated, or disaggregated?
Do things need to happen, like transcribing interviews,
or entering survey responses into a spreadsheet
so the group can easily examine them?
This also would include maybe producing some simple visuals,
like a bar graph, or infographic, to help the group digest the data.
And then last, but not least, in advance of the meeting,
preparing and distributing the agenda.
And depending on the group and the objective,
the agenda may be distributed before a meeting
or at the start of the meeting.
And then we move to during the meeting.
So, again, this is the heart of the protocol, really digging into the data
and having the discussion with the group that you've assembled.
So, starting out with introductions,
making sure the group is aware of the evaluation questions,
the scope.
This is an opportune time to give context
and any additional information to the group
that might be helpful if they're considering the data.
And then also, this is a time to preview the steps of the protocol.
So, again, using the protocol as a roadmap for the group,
letting everyone know where the group is headed
in terms of examining the data and answering the questions.
The next step in the meeting, present the data.
So, all of that work you've done before the meeting to pull the data together,
and format it for presentation to the group would be done
after that introduction of the objective.
And then we get into the discussion of the data,
and you'll see there, that three through five are discuss the data.
And in just a moment, on the next slide, we'll dig a little deeper into that,
because we have parceled those steps out in a little bit more detail.
And then the last couple of steps are to determine the next steps for the group,
which might be specific actions or additional meetings,
and then finally to give the participants
an opportunity to reflect on the meeting's effectiveness
and to give the facilitator feedback about what went well
and what didn't during the meeting.
So, let's go back and dig a little deeper into discussing the data.
So, in the protocol,
we outline discussing the data
in three distinct steps,
steps three, four, and five.
And I think a lot of our tendencies are to, when we see data,
we often jump directly from implications.
And so we broke this down in the protocol to encourage groups
to really observe the data before moving on
to interpretations and implications.
So, we encourage the group and the facilitator
to lead the group through discussing the observations.
So, asking questions like, "What do you see?"
What do group members see in the data that you're presenting?
What are initial thoughts or reactions?
What information does this data not provide?
So, getting participants to think about what is not shown,
what are the limitations of the data that's being presented?
And asking other questions, like, what surprises
you about what you're observing from this particular data set?
And then moving the group from those observations to interpretations.
Getting the group to talk about, what did the data tell you?
What answers are you getting to the original evaluation questions
or the original objectives that you post to the group?
What do these data confirm?
And again, getting the group to think about,
are there perspectives we haven't considered?
Are there other data that might be helpful in answering our questions?
And then finally, in the discussion of the data, moving to implications.
So what? Why does this matter?
So, we've made observations, we've made observations.
Now, what does this mean for our work?
And so leading the group through each of those steps
to really examine the data,
and then think about how it impacts
the initiative or improvement effort
we may be looking at for that particular meeting.
So, we put together just a simple example here of a data infographic,
a chart that you may look at in a meeting.
This particular one is in regards to family engagement.
And the question that was posed here was,
has family engagement increased over time?
And we're assuming, too, here that the group that's seeing this slide
has some context and information for it.
But when we look at it, we can see there are a lot of observations,
there are a lot of interpretations and implications
that we could discuss,
but we'll just talk about kind of one of each today,
just to give an example of how this protocol might play out
when you're actually working with a group.
So, for example, a group may be looking at this
with the evaluation question that's been posed.
Looking at the percentage of families within each district
who report active engagement of their child's school.
And we can see at the top that those bars have been identified
as the dark green being spring 2016 data,
and the lighter green, grey color as being spring 2017.
So, an observation of the data that we might make as a group could be,
one observation could be district one reported a 70% engagement in 2016,
and an 85% engagement in 2017.
We also might make the observation
as a group that two of the other three districts
also saw an increase in family engagement
from 2016 to 2017. Those are just observations.
They're simply just stating what we see in the data.
We then would hopefully move the group to interpretation of the data.
So, now that we've looked at the data,
and made some statements about what we see,
we might move to interpretation.
And interpretation might be overall family engagement
has increased from 2016 to 2017.
And then we would move the group to implications,
and discussing implications of the data,
and this group may decide,
one implication might be we need more data
to determine why districts one, two, and three,
and seen an increase in engagement,
and why district four has not seen an increase in family engagement.
And so the next slide just summarizes those observations,
the interpretation and implication I just shared.
We wanted that to be available in case
someone's downloading the presentation,
or if you want to share it later with your colleagues,
you'll have it there as part of the slide deck.
And while this is a simple example,
it gives you an idea of how you could lead a group
through this part of the protocol
to really give the group time to truly observe the data
before moving to interpretations and implications too quickly.
And then the last piece of the protocol is after the meeting.
And these are pretty straightforward, clear steps,
but we think they're great reminders
for a protocol lead and facilitator,
and the participants as well to keep the process moving forward,
so that the progress that you make during the meetings doesn't end there.
So, we just have some reminders about distributing
the notes that the note-taker
will take throughout to the group,
and then also confirming those next steps.
So, as part of the protocol,
we want the group to identify those next steps,
after the meeting, confirming those steps,
and also the timeline for any additional actions
that might have been identified during the meeting.
So, that is a quick overview of the protocol,
and now I want to turn it over to Debbie Shaver.
Debbie's going to share some examples from our pilot of the protocol.
>> Thank you, Kim.
Well, we are very grateful to two states
who were willing to work with us during a pilot test,
so we could see how this process works
in real places
with real people, looking at real data.
So, the first example was New Mexico Part B,
and I'm going to set the context for the discussion,
but then I'm going to invite Marsha Allen Martinez
to join us and share her reflections
as a state participant in this process.
So, first, to give you some background, New Mexico Part B,
their SSIP focus is on improved reading achievement
for elementary school students
in their results-driven accountability schools,
which are their SSIP schools.
The evaluation question that they selected to focus on was,
"To what extent are the RDA schools
implementing the evidence-based practices that are expected to result
in improved outcomes for students with disabilities?"
In their case, they used data that was collected from their site visits,
and they have a very well-defined rubric
that measures practices across several domains.
They go on-site and collect information
through interviews and observations.
And so that's their data source.
It's a lot of data, a lot of source of great discussion.
In the New Mexico example, the state staff and myself,
as the IDC TA provider,
we worked together to serve in the role as the protocol lead.
So, we worked together to identify the evaluation question,
the data source, who should be at the meeting,
and do all the planning together.
And then at the meeting, there were two meetings,
I facilitated the data meetings as an outside facilitator.
In their case, they had two meetings.
The first one involved only state staff who were quite familiar with SSIP,
and their second meeting had a wider variety of participants,
and it included some of the people that were in the first meeting,
but it also included additional stakeholders,
such as some principals, some other state staff
who were not as involved in SSIP,
and one of their professional development providers.
With that, I'd like to invite Marsha
to share what it was like from their perspective.
And she is from New Mexico's Public Education Department.
>> Thank you, Debbie.
We were so excited to get to participate
with Debbie and Mary, our consultants, on this data discussion protocol.
I don't know if any of you are like me,
where you've been in a meeting,
and you're going to talk about data,
and you're talked to, and just presented some data.
Or if you've ever been on the presenting end,
where you spit out the data and then looked at everybody
and just thought, okay, it's over,
and you don't feel like you had a real discussion.
And sometimes we find ourselves wondering,
with our vast amount of data, what are we going to talk about?
How are we going to get down to the nuts and bolts,
and figure out what is working, what is not working,
and what we need to do?
And so we were really excited to get to participate
in this pilot of this protocol.
And on day one, as Debbie discussed, we were with staff from our office,
looking at data around fidelity of implementation
of our evidence-based practices.
And we had a really great conversation that day.
I will tell you, Debbie did facilitate,
Mary was taking notes, and I feel like, for the first time,
we had a true conversation about what our data meant,
and what the implications were for the future.
It was a process that was very stair stepped.
We took a broad look, and then we started diving further into the data.
And so it was facilitated
so that conversations were had
that we could really reflect with one another.
We could really be honest with one another.
And somebody might say something that led
to somebody else thinking of something else.
And by the time the day was over,
I feel like we had had the most honest conversation about data
that we ever had in our office.
And it was really exciting for all of us.
And we were very interested to see what day two would bring
when we brought new people in.
And so, the second day, we had, as Debbie said,
some principals from some of our RDA schools.
We had one of the loads for our coaching piece
that we do in our program.
And it was very interesting to see that very different things
came out of the analysis of the same data.
Those people have a different perspective.
They are living the data in a very different way.
And so we were able to have really honest conversations with them
about the implications of just being a part of the RDA,
and implementing it in the schools, and how that felt, and what this meant.
And so, two days of looking at the same data
gave us lots of information and lots of steps
that we need to take moving forward.
And we have a very clear vision of what we need to do with that now.
Because of that experience, myself and Felice,
who is my partner in crime in the data worlds for RDA,
we have decided that we are going to use this protocol actually this Friday,
and we are going to go out
and meet with about half of the principals in our program,
and we are going to look at student achievement data,
and have a real clear conversation
and a supported conversation with them
about overall achievement in our state for students in RDA schools
who are students with disability,
and those who are not.
And then we will present again next Friday
with the other half of our principals.
And we are really excited to see what this protocol
is going to be able to do for our meetings with them.
In the past, sometimes we've left there not feeling like it
was a real productive meeting,
because we didn't have a clear path
of where we were going with our discussion on data.
And so we were really excited for moving forward
in how we are going to be able to facilitate these conversations.
We also have plans to use this piece
with our idea stakeholder panel in March,
and get information from our stakeholders as well.
And so we're really excited,
and we would encourage you to get in there,
and practice this, and look at this. It is great for your staff.
It's great for your stakeholders.
It's great for just honest conversations.
And look forward to hearing what other people
are having happen with this protocol.
Thank you.
>> Thank you, Marsha, that was a very enthusiastic report,
and I'm so glad to hear that you are taking the protocol
and using it in different ways.
So, that is really great.
All right, I'm just going to share a little bit about another example,
just to give you another idea of how it was used.
New Hampshire Part B, the preschool program,
also helped us with the pilot test.
And their SSIP focus is on improving social and emotional
outcomes for preschool children with disabilities.
And in New Hampshire, excuse me, in New Hampshire
they had several objectives
in using the data meeting protocol.
The first one was around practitioner fidelity of implementation,
and looking at what the status of that was,
as well as what the implications of their findings
would be as they moved forward and tried to improve that work.
And then secondly, they wanted to look at data about looking at the process.
What was the value of the process?
What additional supports did people feel were needed?
And they used two data sources.
The first was their fidelity of implementation observation tool.
And the second was a survey
where they collected feedback on their process coach.
In this example, the state staff took the lead as the protocol leads,
with support from the IDC TA provider,
and that was Tamara Nimkoff.
And so they took the lead, and she supported them,
and then the state and local staff
did the facilitation of the data meetings.
And they, too, had several meetings.
The first two were with their state leadership team,
and their third meeting involved local leadership
and implementation teams.
So, that's just to give you several ideas
on how it's been used in the past,
and Marsha shared some ideas on how they plan to use it in the future.
We are very grateful to both of those states for doing this.
We had a draft version of the tool at that time,
and we were able to tweak it
and improve it based on our experience in those two states.
Well, today's purpose was to introduce this tool
as something that could be used to have authentic,
but yet organized and systematic discussions about data
that hopefully will improve states' ability
to really use all of that data that you are collecting,
and use it in a way that's really going to help move you forward.
So, again, the data meeting protocol
is a simple way to conduct data discussions.
It can help you assess progress toward achieving intended outcomes,
and that could be in SSIP, or in any other program,
or improvement effort that you are focused on,
and can inform the next steps in SSIP implementation.
So, as people discuss data,
it's not just data sitting out there,
you know, that you're reporting,
but it's data that you are using to continually improve your work.
There is a link here to the actual tool.
The tool was just posted in its final form just before the meeting,
so it is there on the IDC website.
And if you don't get a chance to copy that link down,
you know, let's put it in the chat so people can copy it.
Sophia, if you could do that.
But also, you can just go to the IDC website
and look under resources, and you will find it there.
We wanted to point you to some resources,
and while I talk about these resources,
I want to mention that we do have plenty of time for question
and answer following the presentation.
And if you have a question, please put it in the chat,
and again, put it to all--
I'm sorry, all participants, and we will do our best to answer questions today.
So, while people are thinking of their questions,
so let me tell you about some resources to help you with using this tool.
The first is individualized TA support with the protocol.
And to get that, please contact your IDC state liaison.
That TA can come in a variety of forms, depending on what people want.
You heard examples where the TA facilitators facilitated the meeting,
and examples where we provided support in preparation for the meeting,
but we didn't actually facilitate the meeting ourselves.
Or, we can just walk you through the protocol,
and, you know, make sure you understand it,
feel confident to use it.
Then, secondly, we are working on a facilitator's guide
to go with the protocol,
for using data with a variety of stakeholders.
The facilitator's guide recognizes that different groups
are going to have different levels of comfort and experience
in having these kinds of discussions.
And so the prompts and guidance for a team,
you know, let's say a state team
that is intimately familiar with data sources,
and data systems, and knows, you know, a lot about the data,
those prompts and guidance would be different than if you were using,
or you were having a discussion with a group
that was not as familiar with the data sources,
or even talking about data.
So, we have lots of suggestions and guidance
for different kinds of groups
and their level of comfort and familiarity
in having these kinds of discussions.
So, those are not ready yet.
We are finalizing those,
and we will post those as soon as they are ready,
the facilitator's guides.
And then for our Part B colleagues, I just want to mention the IDC
Interactive Institutes that are coming up in February and March.
The theme of those institutes is around building
a culture of high-quality Part B data.
For Part B people, these is an on-site, in-person meeting institute.
The registration is in two days,
so if you haven't signed up already,
we encourage you to do so.
And you can contact IDC for more information
about those if you need them.
At the institute, we will be holding a session
that goes into this protocol in more depth.
That concludes our presentation.
There's the IDC website.
Sophia has put the link to the actual protocol in the chat box.
And Tamara has mentioned that the slides
and the recording will also be posted on the IDC website for you to access,
or for you to refer others to.
Again, we are grateful to the two pilot states
that helped us make final changes to improve the tool.
The tool is up.
As you heard from Marsha, from New Mexico,
it was really valuable in their case,
and we hope that it will be a tool that other states can use
to have really great discussions about your data
and, you know, really use that data to improve your work,
and inform what you're doing.
>> Okay, there is-- so I would say the presentation has officially concluded.
Yes, the slides will be available on the IDC website.
And then there's a question,
are either of the implementation fidelity measures
that the pilot state used available as resources?
I don't know the answer to that.
We will try to get that--
>> I can speak a little bit for New Hampshire, Tammy.
New Hampshire, there, the particular measure that they used
was an already-produced measure.
It was, they are doing,
using the pyramid model
with their preschool providers.
And so it was the teapot.
It was, you know,
commercially produced
by the provider for that.
So, that was what New Hampshire used.
They used the already produced reliable
and valid measure by the provider,
called the teapot. And then--
>> And then, New Mexico, they developed their own site visit rubric,
and they've been improving it over time,
and we would have to check with them to see
if they are sharing that with other places.
It's very tailored to what they are trying to do in New Mexico,
but we'll try to find that out and get back to you.
>> Sure. And if you're interested, Tammy, in talking more about, you know,
possibilities for measuring implementation fidelity in your state,
then please, you know, reach out to your IDC state liaison.
If you don't know who that is,
then right on the IDC website there is a link to find your liaison.
And we can coordinate in looking at possible measures
that would be appropriate for what you guys are using.
>> Well, hopefully the protocol is so clear
that even when you look at it, it'll be clear on how to use it.
But again, we are happy to support you in any way
that you think would be helpful.
Tamara, any final words?
>> Absolutely. No, just telling [INAUDIBLE]
yeah, feel free to email. That's great.
Thank you very much, everybody.
And again, thank you, Marsha, for sharing,
and to my colleagues, Debbie and Kim, for sharing in the presentation.
And we look forward to supporting everybody
and hearing from everybody. Have a great day, everyone.
For more infomation >> Battlefield 1 - ARISAKA TYP 38 Waffen Review ► BF1 Guide (DLC Waffe) - Duration: 3:35. 
Không có nhận xét nào:
Đăng nhận xét