>> LAURA ARON: Welcome to understanding WIOA disability reporting, tools for data visualization. We are so happy that you're here with us today. Today's webinar has two hosting projects. The first is the LEAD Center which stands for Leadership for the Employment and Economic Advancement of People with Disabilities. And we are a Workforce Innovation and Opportunity Act or WIOA policy development center of the Office of Disability Employment Policy. Social Policy Research Associates and National Disability Institute lead the LEAD center. Our second host is DWQI or the Data Workforce Quality Initiative. It helps states build better connected data systems to better serve customers. Social Policy Research Associates leads the DWQI project. So that everyone can fully participate in today's webinar, we'd like to take a moment to share some captioning and housekeeping tips. Today's webinar is live captioned. The captions appear below the slide deck. You also have the option to open the captioning web page in a new browser. The link has been posted in the chat box. You can adjust the background color, text color, and font using the dropdown menus at the top of the browser if you choose to personalize your captions. Just position the window to sit on top of the embedded captioning. If you have content questions during this presentation, and we do encourage you to ask them, please type them into the Q&A panel and we will save time at the end of questions and answers. If your question is not content related, you can just type it in the chat box. Lastly, if you're experiencing technical issues or have questions for the technical support team, open the participants list and select the raise hand button next to your name. To kick off our presentation today, I would like to welcome Melissa Turner. Melissa is the Director Of Special Projects in the Office of the Assistant Secretary of the Department of Labor's Office of Disability Employment Policy or ODEP and she's the project lead for the LEAD Center. She previously worked on ODEP's employer policy team leading work with employers to increase recruitment, hiring, retention and promotion of individuals with disabilities in the workplace. While she was at the Office of Management and Budget, she developed regulations to protect the rights of individuals with disabilities under WIOA and the Individuals with Disabilities Education Act. Melissa, welcome to the webinar. >> MELISSA TURNER: Thank you, Laura. Welcome to "Understanding WIOA Disability Related Reporting: Tools for Data Visualization." I'm Melissa Turner from the U.S. Department of Labor's Office of Disability Employment Policy or as we call ourselves, ODEP. ODEP funds the National Center on Leadership for the Employment and Economic Advancement of People with Disabilities or LEAD Center which as Laura said is hosting today's webinar along with the WDQI. They gathered excellent speakers for today's presentation. ODEP and LEAD are so excited to share the new WIOA data visualization tool with you today. The project really started from a place of curiosity. We wanted to know more about progress in reporting on the disability data points required under WIOA. We worked with the LEAD Center to analyze the data in the hopes of answering a few key questions. What are states reporting? How much are they reporting? And what stories do these data tell us about the ways that people with disabilities are served by the workforce system? The presenters today will share more about the resource and we hope you find the tool to be informative as you continue your work on disability inclusion. We will also be joined by speakers from the Wisconsin workforce system to discuss effective reporting and analysis of WIOA disability related data and how it can enhance services and help mitigate job losses for individuals with disabilities. Before I turn it over to our presenters, I want to take this opportunity to let you know about two exciting anniversaries that are happening this year. ODEP is working on a year-long series of events and resources to commemorate the 30th anniversary of the Americans With Disabilities Act or ADA which was signed into law in July of 1990. Additionally, this October marks the 75th anniversary of Disability Awareness Month. We invite you to learn more during this important year on our website at DOL.gov/ADA30 and DOL.gov/NDEAM. With that I'll turn it back to Laura to help us dive into today's content. >> LAURA ARON: Thank you, Melissa. What a great introduction. So my name is Laura Aron. I'm the project manager of the LEAD Center and a senior associate at Social Policy Research Associates or SPR. Today you will also hear from Joshua Mallett, senior data programmer at SPR and Bruce Palzkill and Bryan Huebsch. Today we are excited to share a new online interactive tool that will allow you to explore your disability related WIOA data, highlight promising practices from the field to help you improve your reporting and ultimately services for people with disabilities and discuss how you can use your data to drive system improvements. One of our goals for this webinar series and really for our larger projects is for states and local areas to know your disability related data. Most states and local areas are not necessarily aware of their data related to customers with disabilities. So we built a tool to easily give you access to your data. Once you see your data, we encourage you to investigate what might be driving your comprehensive or not as comprehensive reporting rates. Some things to look a could be training of staff. Do staff understand why we collect disability related data? Not just how to input the data. And other avenues of investigation may lead you to your case management system and how customers and staff interact with intake and registration forms and/or you may want to look at your current outreach to people with disabilities. Once you have a better idea of what might be driving your reporting, you can develop a plan at the state and local levels. Bottom line, if staff in the field can better understand who they are serving, their programs can better connect with and serve people with disabilities. I want to share with you some of the results of our national data investigation. Basically it shows that overall there is low reporting on most disability related data elements. For example, about 74% of participants with disabilities do not indicate their type of disability. There are a few states though that have high rates of reporting across most disability related data elements including Wisconsin who, as you know, are with us today and they are also with us next week for part two to share their promising practices. Lower disability related reporting rates are likely due to a misunderstanding of what many data elements intend to capture and how these data can help both programs and stakeholders trying to understand the challenges faced by people with disabilities. I would now like to turn it over to my colleague Joshua Mallett who will show you quite a nifty data online tool. Joshua? >> JOSHUA MALLETT: Thank you, Laura. Hello, everybody. As Laura said, I'm Joshua Mallett and I'm going to give a nice overview of our interactive data visualization tool we built to help you explore WIOA reporting disability related data elements. A fantastic aspect of the tool is you will have the ability to explore these rates within your own states, at the county level, as well as across the six ETA regions. You have been provided with the link here which you are welcome to use and explore at any time. This tool is available online as Laura mentioned. I'm going to walk through how to use the tool today and all the various tabs and pieces of information that you can explore deeper at your own leisure. So first let me switch over to the tool. When you come to the home page of the tool, you'll immediately be presented with a map of the United States. You'll see that the map is broken down at the state level and there's a lot of things to explore here. First I'm going over the main four tabs you'll see at the top of the screen that I'm sharing. There's the welcome tab in the top left corner of your screen where you are presented with an overview of how the tool works. There is also the data by map tab which you're presented with here that has four various sub tabs that I'm going to go into detail on in a bit that show maps for all the disability related items in a variety of ways. There is the data by tables tab which is not only a fantastic way to get a deeper look at the rates of reporting for all of the disability items as well as the population counts in all of these areas, counties, and regions, but it also allows for a heightened level of accessibility for those who may not be able to view the maps visually. Lastly, we also have the data guide which I'm going to go into a little detail later as well, where you can explore the ten disability related data elements and their definitions and short summary descriptions of what they mean. So I'm going back to the data by map tab just to start out to go through each of the sub tabs briefly to give you an idea of what you can explore here. The immediate tab you are brought to underneath data by map is the disability status tab. Underneath this tab as I said before you are presented with a map of the United States where you can explore the reporting rates at the state level. Yellow indicates higher levels of reporting and purple indicates lower rates of reporting. You can see next to the map there is a legend that provides the range of these rates of reporting as well as a summary of the overall reporting for the United States across different programs. Additionally, there is the disability status by program tab where you can explore rates of reporting by programs including the WIOA adult program, the WIOA dislocated workers program, the WIOA youth program, and the Wagner Peyser and grants participants program. Under the new under WIOA tab you are able to view additional disability related elements. As you may have noticed on the disability status tab you are able to explore the first disability related data element. This tab allows you to explore the nine additional disability related elements that are new under WIOA. Again, you can explore these for all states with a legend that reports the yellow shading for higher rates of reporting and purple shading for lower rates of reporting. Lastly, there is also the state comparison tab where you'll be able to compare your state to see if your state has more or less comprehensive reporting compared with the other states in the U.S. on average. Before I go into some of the results that we have found across some of these tabs, I wanted to talk about each of the disability related data elements briefly. So, again, if you go to the data guide tab you're able to look at a table of each of the ten data elements. In the first column you can see the name of the data element and then a definition and description. I'll briefly list the elements and please feel free to explore them. The first data element is individual with a disability. Also categories where individuals indicate that they have a disability. You can also look at state developmental disabilities agency services. You can also see the local or state mental health agency services. Medicaid Home and Community Based Services waiver services. Whether an individual has indicated their work setting, their type of customized employment services received, financial capability, individualized education program and lastly the Section 504 plan whether or not an individual has this or not. So I'm going to move back to the data by map tab and go back to our disability status tab. As I said before, we are seeing a map of the U.S. At the top of the map you can see that we have a short summary that describes the overall reporting rate where we see that overall 5.2% of WIOA, Wagner Peyser and JVSG (Jobs for Veterans State Grants) participants indicated having a disability. The rates of reporting are shown in yellow and purple where yellow indicates higher rates of reporting. You can see here as well that on their left you can search individual states where you want to investigate reporting in your state at the county level to better understand the rates of reporting within your state. I'm going to type in an example. You can type in a state abbreviation or the full state name. When you type that in, the map will update below and you can see your state, rates of reporting at the county level. So, for example, here you can see Idaho reports 13.1% of WIOA, Wagner Peyser, JVSG participants indicating having a disability. Scrolling back up here, an important part about this tab is not only that you can explore the reporting rates of participants who indicated they have a disability, but also understanding what that means as well as what it means when an individual does not report their disability status. We have a nice graphic here where we just wanted to explain a little deeper what this means. So, as you know, when you report your disability status, when a participant reports this they can indicate that, yes, they have a disability or they can indicate, no, they do not have a disability. There is an option to not indicate whether or not they have a disability or they do not have a disability. This is something our online data visualization tool explores to better help us understand where reporting rates are far lower. Again, this can be seen across the U.S. I'm going to go back to our tool here. You can see on the left side of the map there is a filter "view by area." This is the area where, again, you cannot only explore at the state level but also at the ETA region level. By selecting under the disability status selection you can switch the map to now explore the reporting rates where individuals have not indicated whether or not they have a disability. In other words, participants' disability status is unknown. The overall summary at the top of the map is updated to indicate 11.5% of WIOA, Wagner Peyser and JVSG participants disability status is unknown. We can see the reporting notes. One thing to note is while we have changed what the information that we are looking at, the yellow shading still indicates higher rates of reporting. So here we can see when purple shading is indicating lower rates of reporting. Again, you can explore any of the states at the county level. We also wanted to show reporting at the program level. As I stated before you can explore this in the tool and we have provided here a table that breaks down reporting rates for individuals indicating they have a disability across each of the main programs in the tool as well as the percentage reporting rates for those not indicating their disability status. You can see that the reporting rates for the WIOA adult program, dislocated worker program and Wagner Peyser/JVSG programs have higher rates of individuals not indicating their disability status. You might also note the youth program has a lower rate of individuals not indicating their disability status. One last thing we'd like to note as well that some of you may know that participant rates may overlap as participants may be involved in more than one program. This information can be explored in the tool by going to the disability status by program where you can look at each of the individual maps for each of the programs and explore them side by side. Lastly, I want to take a look at the new under WIOA tab because this explores each of the nine new disability related items. Again, you can see on the left filter view by, you are not only able to change the type of map you're looking at but you can look at each of the nine disability related elements beyond disability status. When you are first brought to this map you can see the category of disability that's been indicated by a participant who has already indicated that they have a disability. As Laura mentioned earlier, 73.7% of WIOA, Wagner Peyser and JVSG participants have not indicated their category of disability. This is another way to help us better understand what reporting rates look like in the U.S. Again, any of these additional items can be explored by selecting them on the left side of the map. So once again we wanted to provide you with the link to this online data tool that you can access at any time so that you can better understand how comprehensive your reporting is. We hope you find this very helpful in investigating the reporting rates in your state. Laura, I'm going to hand it back to you. >> LAURA ARON: Thank you, Joshua. It certainly is eye opening when you see your own data. Hopefully it will propel state and local staff to replicate promising practices they see in their own state as they investigate their data and also diagnose what might be happening if the data is not as comprehensive as it could be. We now turn to our colleagues at the Wisconsin Department of Workforce Development to share their promising practices related to disability related reporting and improving services for people with disabilities. I'll turn it over to Bruce Palzkill. >> Thanks, Laura. Good afternoon, everybody. Before I turn it over to our colleague Bryan, I wanted to just paint the landscape of Wisconsin a little bit about our partnerships and our job center structure and that helps guide or influence the presentation that Bryan will be doing. Wisconsin has been doing our integrated job centers for over 30 years in most areas of our state. And with key partners as part of our system and more partners coming along the way to join us within these integrated locations, I think that's a critical piece because we have had a longstanding situation where both customers and our partners are just used to working together, collaborating on referrals, on intake, collaborating on cross training of the partner programs between the titles. In Wisconsin a WIOA state plan we have the traditional four titles plus JVSG and TAA (Trade Adjustment Assistance) and this year we have added on the TANF, FoodShare, and reentry programs. Long before those three programs joined us we have had a long history of collaboration with them as well. We don't have everything in one reporting system, however we do have very good collaboration going on amongst them. I think some of the key things I wanted to share was that this world of partnerships takes a long time, but we see that it pays off in trust with our customers. It's all they have known with your job center system for the better part of three decades is coming in knowing there are a bunch of agencies working together under one roof trying to avoid duplication of service, trying to figure out what are the right services our customers need including those with disabilities and really trying to have our job centers be as effective as they can be. So we are very proud of our job center system and very proud of our integration that we have amongst the partners. Very similar, I'm sure, to what the experience is in many states, but it's something that we have worked on for a very long time. With that I want to turn it over to Bryan Huebsch who will walk through the data he's presenting for the state of Wisconsin. Bryan, take it away. >> BRYAN HUEBSCH: Thank you, Bruce. Over the previous six years I have worked with WIOA performance and reporting. Our state agency has been on a journey to evolve its data collection strategy from one of just meeting performance measures to actually evidence building activities. Nobody woke up in 2014 following the enactment of the Workforce Innovation Opportunity Act in Wisconsin and said, Wisconsin is going to be lights out at reporting data elements regarding disability. Instead, we thought we needed to make the data collection processes provide more value to customers who receive our services. That's where we focused. That work continues today despite my reassignment to a more global data management position in the department. One of the beliefs was to provide context behind asking the questions that comprised the participant individual record layout known as the PIRL or the federal data collection requirements. The required collection that must be asked of our participants is vast. We were concerned about first receiving truthful responses and then second the impact and the potential for decreased staff morale. Staff did not enjoy entering data into a computer system. It's not why they signed up to work with us. This dissatisfaction is often exasperated when staff do not perceive there is a good purpose to collecting that information. So I believe there are two ways to get a person to answer any question truthfully and completely. First, provide the context for the collection. Second, trust the answer will not be used inappropriately. This additional information provides purpose, whether the staff or the customers agreed with the purpose. It forms a type of legitimacy instead of leaving the client or customer to wonder what the government is doing with my data. Next slide, please. I do not have any proven explanations why Wisconsin customers answered these questions at greater rates than other states. After initial discussions with Social Policy Research, our state learned that all the common sense things such as making the fields mandatory in the system were not unique to Wisconsin. There must be another explanation. So I'm going to provide an approach and a way of thinking that's been gaining traction within Wisconsin's department of workforce development and our workforce system in general. The approach is increasing the importance of data management concepts such as data governance and data stewardship. If you read a lot of books, you may become smart and well educated in something. If you are a football team and you practice tackling a lot, you might end up having a good defense. So if we have good data management practices, we may be good at reporting disability related data despite not deliberately focusing on that specific goal. The WIOA partner programs in Wisconsin have developed an integrated service delivery governance structure that's focused on driving integrated activities between the WIOA programs. This structure developed recently in 2017. The state also received a workforce data quality initiative or WDQI grant during round seven that is designed to build the governance structure and data sharing capabilities to help WIOA and many other workforce programs in the state to meet their evaluation requirements. These newer structures and initiatives are designed for the department and its partners to be set up for long term success in collecting and using data. Next slide, please. There are obvious benefits for workforce providers when complete data is collected. These benefits provide the motivation for the providers whether they are local workforce board contractors, our department's Bureau of Job Service or Office of Veterans Employment Services to assist workforce clients best to provide this information. When reviewing paper applications, our staff assist by explaining the purposes for and reassure them that their data is secure. This is also highlighted in many of our release of informations that our clients must sign. I will highlight the impacts of the statistical adjustment model on the performance goals at the end of the year in another webinar. I believe our state was successful in messaging to workforce provider how important reporting all the characteristics of the participants served and how economic conditions impact end of year adjustments mandated under the workforce innovation accountability provisions. Additionally, the state and local areas benefit by enhancing their evidence based activities by having enough and accurate data that it increases the effectiveness of research and evaluation activities that can be done. Operational reporting also tells providers information about the scope of their activities. Everybody likes to get good feedback about what they are doing, whether it's a score board of your favorite football team, your report card in school, or information about something else you have interest in. Everybody likes to see good, positive feedback. This is no different for our workforce providers. Additionally, those who wish to excel, they don't mind finding out if there are some of those opportunities to improve. These are all motivations to collect data completely and accurately. Next slide, please. I alluded to this earlier. But it is important to let customers know why we collect all of this data. Beyond the obvious reason, because it allows them to access program services and it is a requirement, the customers who access our system benefit from having informed staff who work with them and can work with them from a position of just being informed about them about the customers. This first bullet leads to the next three which are all good for the customer. Informed career planners can do a better job of assisting the customer on their journey to appropriate employment opportunities. There is more flexibility to inform opportunities that have maximum resources to meet the highest number of needs the customer has. Lastly, customers enjoy informed referrals because nobody wants the run around of an inappropriate referral. Inappropriate referrals are frustrating to the customer and make the staff in the workforce system look disingenuous and not knowledgeable. Our data steward team did some training in a common intake project about a generic customer named Jordan who experienced both appropriate and inappropriate referrals. It was kind of like a choose your own adventure book. The range of emotions Jordan experienced were from extreme relief that she was receiving assistance to extreme frustrations from an unproductive day at the job center. That's the training we gave a number of staff to help illustrate the importance of informed referrals and how that impacted our customers. Next slide, please. Employers also benefit from this information. However, I want to point out that employers would never receive this aggregate information from our programs for any customer. The workforce system models behavior that is required by the Americans with Disabilities Act. Employers also benefit because the workforce system assists employers by showing them how they can meet their critical workforce needs by hiring individuals with a disability. The business service teams are trained to help alleviate concerns employers have regarding individuals with disabilities and how that may impact their operations. All of these efforts lead to improved employment outcomes for individuals with disabilities while meeting business talent needs which ultimately help meet the needs of our state's economy. Next slide, please. Wisconsin has integrated American Job Centers. This enables good communication between programs and partners. The state's labor market exchange, the Job Center of Wisconsin.com website is often the first the point of entry for most customers who seek our program services. The initial registration questions on JCW (Job Center of Wisconsin) are standardized to meet the reportable individual requirements set out by the United States Department of Labor. Additionally, the reemployment services and eligibility program has an online questionnaire for all participants. The RESEA (Reemployment Services and Eligibility Assessment) program has been fully integrated with the WIOA Title III Wagner Peyser employment services program for many years now. The same agency, the same people, the same data collection system. The disability related data elements are on the RESEA questionnaire and get collected into the intake and are immediately shared with the Title III program. The Title III Wagner Peyser and RESEA programs typically have the largest volume of customers in our system. Let's go to the next slide, please. Wisconsin's vocational rehabilitation program is in the job center and at the table in the one stop operating system. Similarly, there is to the employment training programs there is participation in state wide committees and executive teams by the division of vocational management. DVR (Division of Vocational Rehabilitation) is a great partner because of the programming and staff experience it brought to Wisconsin's public workforce system. Although we do not have a real-time data sharing with the division of vocational rehabilitation, having their presence and helping in informing a lot of the activities of the employment training program, we find that very beneficial. Next slide, please. The programs integrate and coordinate activities in the job center so they can most effectively leverage the resources available to maximize customer benefit. The I word can be very scary for many workforce programs. We believe this is often due to misperception. We integrate the program activities with each other. When our programs work together the common customer wins. Program coordination and integration is done well since many employment training programs already use the asset system, the information source is the same for these programs that I will show you on the next slide, please. The Division of Employment Training which administers the WIOA Title I, III, and Wagner Peyser Jobs for Veterans State Grant and other state programs has an integrated application to collect participant data. The automated system support for employment training has been in place since 2003. Also known as ASSET. It was initially built to meet federal reporting requirements of the Workforce Investment Act of 1998 program. As a growing desire to know what's happening in the programs and the evidence based movement, ASSET was and has continued to be asked to do more and for more programs. The system integrated more federal programs during the transition from WIA to WIOA. Recently, the trend has been to put more state specific programs into ASSET to first avoid the cost of having an independent application and all of its unique costs. Second, to also improve the integration between the other programs and the coordination. The Wisconsin summer youth program is funded by a provision in state statute. There are several programs that grew up in separate systems but are becoming more integrated through projects like the integrated service delivery project I mentioned earlier. Next slide, please. Even before the pandemic, communication with staff was primarily done through webinars. Consistent messaging regarding data handling and performance topics were delivered through the WIOA performance Friday webinars. In addition, DWD (Department of Workforce Development) created content specific video tutorials that can be accessed at any time by staff whenever it is convenient for them. Webinars ended up getting became a gift that keeps on giving as it was a venue to help develop an understanding among staff and could be used in multiple times in multiple venues. You can see a few of the topics routinely addressed with staff on this slide. I will highlight the case management system changes that were frequent following the introduction of WIOA. Those changes were made to comply with new and modified reporting requirements. It is easy to confuse staff members when the system they log into every day is being updated in an incremental fashion. So timely communication was necessary to assist staff to stay on top of their management system. These webinars happen two times a month and are recorded for future use in technical assistance visits and on demand by staff. Next slide, please. This slide talks about some of the topics that were recently delivered and are directly relevant to today's discussion. Training specific to equal opportunity, nondiscrimination, and confidentiality have some impact on serving and collecting data from individuals with disabilities. The disability related reporting was covered in one of the earlier webinars. Also often business services are discussed with materials that come from the division of vocational rehabilitation. So the webinar series has provided presentations to the business services team to help the business services individuals with disabilities. So it's a very diverse webinar series that's designed to help out on a lot of things that can contribute to increased and improved reporting. Next slide, please. The targeted outreach is informed by data on which areas may have potential workforce and potential, you know, employers that can help support the state goals for the department. Labor market information from our website Wisconomy.com where labor information lives. It is used to develop informed strategies in their workforce plans. The plans emphasize activities and partnerships that assist in serving those with employment influencers that's what we call them. WIOA calls them the barriers to employment. These relationships between the programs help streamline the activities to the individuals with barriers to employment or with disabilities and the other employment influencers as I mentioned. Next slide, please. American Job Centers put out flyers and social media messages about once per month on how the system can help potential customers gain employment. The flyers are often specific to veterans, individuals with disabilities and veterans with disabilities. Flyers are often tailored to I'm sorry. The one stop partners collaborate to host job fairs that invite employers and often times they are able to hire people with disabilities. These programs post these messages to social media in order to promulgate a lot of opportunities to the public as well as on the main website. DWD.Wisconsin.gov. The Job Center of Wisconsin, all of the one stop partners use JCW in some form to assist customers in their job search in their areas. Additionally, widgets have been incorporated to assist customers learn more about the labor market and to provide additional resources to help customers improve their skills. Additionally, there are partnerships locally with agencies who all work to improve employment prospects for individuals with disabilities. In this slide you can actually click on some of those highlighted partnerships. As I mentioned earlier, the results for the state and reporting disability information can mostly be attributed to quality and effective outreach. I hope our presentation helps spark ideas about how to engage and report on individuals with disabilities. Thank you for your time today. >> LAURA ARON: Thank you, Bryan, for sharing your practices and how they benefit individuals with disabilities, employers, local areas and your state. If you have questions for either Joshua or Bryan, please put them in the Q&A panel. We'll get there shortly. Just want to share that, of course, after the webinar we really encourage everyone to play with the online data visualization tool. That link is in the chat. We'll put that in the chat again. It's also here in the resources that will be posted on the website later. And also for more information on the status of disability related reporting nationwide you could access a five page brief on the LEAD Center website. Now we're going to turn to questions and answers. So far we have a couple of questions for Joshua and a couple of questions for Bryan. If you can go to the next slide. Okay. So, Joshua, we're going to start with a couple of questions for you. One guest would like to know how recent the data is on the tool and if there is a lag. I'll just add about when it will be updated. Josh? >> JOSHUA MALLETT: Thanks. We actually list the dates for when the data or what data we're using. So these are to April 2019. For each of the quarters in those periods. I think Laura said you would mention updating this. >> LAURA ARON: Okay. You'll turn that back to me. That's good. Thanks for that question. Yes, we want to update the data. That's just in process. We hope to update it every quarter. But you're correct that just as this data is going to have a quarter lag, because it comes from the states and then is cleaned and then we'll be able to update the data. Thank you so much for the question. Joshua, while I have you, I would like to ask one other question that came from our guests. It asked, can you say more about how and where the data points are collected? >> JOSHUA MALLETT: So I'm not exactly sure beyond that the data is coming through in the PIRL. So each of these elements are items that are asked of participants, the data are recorded into the system and then these data are aggregated and reported here. So these are specifically ten variables in the PIRL system that we are reporting. They're reported at the individual participant level. So we are then, as Laura mentioned, cleaning these data to aggregate them and report them at the county, state and ETA region level. I hope that >> LAURA ARON: Thanks, Joshua. Just for laypeople to know case management systems coordinate with these you know, people are throwing around the word PIRL, you don't need to know any of those names now. Basically when case managers are entering data, all of that data that you enter corresponds with the kind of official data reporting system for the Department of Labor. And so that data is regularly sent by your state. I'm going to move now to a couple and, please, if we haven't answered your question, go ahead and ask again in the Q&A panel. Hope we covered that. Bryan, I would like to turn to you. There are several questions for you. The first: Does your state require a response to all of the disability data elements once they indicate they have a disability? >> BRYAN HEUBSCH: We require a response, but the values are yes, no, and no response. Also known as the individual does not have to disclose. Because this is all voluntary collection, we do require a field to be a response to be entered, but there can also be the participant didn't want to disclose to us. >> LAURA ARON: That's great. It's a great question. We have another question for you, Bryan. Someone asks: What size staff do you have in Wisconsin to support all of these performance related functions? >> BRYAN HEUBSCH: Until about a year ago we had one business staff. That would have been me. Now we have two or one and a half. Mr. Jeff Orr has two jobs. On the IT side, there was a project manager in the division of employment training and then a few developers who would work on this. So it isn't that we have a lot of people who are dedicated to this. The idea was that we would, you know, utilize the staff that are either in a job center or in the program areas by bringing them up to speed on many of these performance concepts so that it would make it more manageable that we only had a few people in our state that were working on this. This was more of a language, more like data as a second language kind of deal so that we wouldn't have to rely on one or two people to do everything. >> LAURA ARON: Thank you, Bryan. I think you've answered the next person part of their question. She wanted to know how you're organized. You have answered that it's now about one and a half people at the state level. She asked further how many people are working on performance data. But then she further asked how many people are working on technical assistance and outreach? Is that another team? >> BRYAN HEUBSCH: So outreach is a hard question for me because a lot of it is done locally in coordination with many of the partners. So I don't have a great answer for that. Technical assistance, we have a number of people who work on technical assistance. So the one and a half, Mr. Aaron Sarbacker, my replacement, and Jeff Orr, they do technical assistance but there is another program section in WIOA that provides that technical assistance to the Title I programs. Additionally in the Bureau of Job Service, I know of a few people that do technical assistance and training for the RESEA and the Title III Wagner Peyser program. In some of our other programs they have one or two people at the state office who work on training for jobs for veteran state grant and in vocational rehabilitation one of the partners has a policy committee that does webinars. So it wouldn't be just one or two people doing all of this technical assistance. It's really spread out across the different program areas. >> LAURA ARON: Thanks, Bryan. Again, thanks, everyone, for your questions. You still have a little time to get questions in. Go ahead and put those in the Q&A panel if you have questions about the tool, the only tool, questions for Bryan at the state level or he might have information about the local level. What we are really hearing from Bryan is kind of consistent messaging. How that's happened is as Bruce first explained about really working on system integration. And having all programs be able to share consistent messaging and down to being able to regularly train your staff. Again, as I kind of mentioned at the beginning as to why you're even collecting this data and not just how you're collecting this data. As you know, a lot of states and local areas do trainings on here's how you input this into the system. But not necessarily why. As Bryan has said, you know, everyone wants to know why. What's in it for me? You know, no matter who you are, whether you're a customer, a staff member, an employer. So as you work in the field, really think about that. Think about who the different customers are that you serve including yourselves and say, well, how do we motivate folks? You know, Bryan really shared that also. So, Josh, I'm going to have you go to the next slide. If any last minute questions come in, we'll answer them. But I just wanted to give a plug for next week's webinar. We hope to see you here. It's kind of the same time. It's part two of this WIOA reporting series. When we get into the nitty gritty of the Participant Individual Record Layout or PIRL. So we'll go through each of the ten data elements that Joshua was sharing and talk about, well, what does it mean, for example, state developmental disability services. What does that even mean? What does it mean you know, what are the responses and what does that mean? So that you are able to provide better technical assistance to your field. I just put the link in to register for next week's webinar in case you still need to do that. Joshua, next slide. If you're not already connected to the LEAD Center, we would love to have you connect with us and sign up to get newsletters and notifications. Next slide. Also, we'd like oh you to follow the workforce data quality initiative, building connected data systems that help us work smarter and deliver for customers. Next slide. Also for WDQI, you can sign up to get their newsletter and sign up to hear their podcast which is super interesting. I don't see any additional questions for our panelists today. So I'm going to thank you for attending today's webinar. We really look forward to seeing you next week. Take care. (Meeting concluded at 2:56 p.m. CT)