Y Pwyllgor Plant, Pobl Ifanc ac Addysg - Y Bumed Senedd

Children, Young People and Education Committee - Fifth Senedd

22/01/2020

Aelodau'r Pwyllgor a oedd yn bresennol

Committee Members in Attendance

Dawn Bowden
Hefin David
Janet Finch-Saunders
Lynne Neagle Cadeirydd y Pwyllgor
Committee Chair
Suzy Davies

Y rhai eraill a oedd yn bresennol

Others in Attendance

Ann Keane Cyn-gadeirydd grŵp gorchwyl a gorffen EOTAS, Llywodraeth Cymru
Former Chair, EOTAS task and finish group, Welsh Government
David Jones Cadeirydd, Cymwysterau Cymru
Chair, Qualifications Wales
Yr Athro Brett Pugh Cadeirydd grŵp cyflawni Llywodraeth Cymru ar gyfer EOTAS
Chair of the Welsh Government delivery group for EOTAS
Philip Blaker Prif Weithredwr, Cymwysterau Cymru
Chief Executive, Qualifications Wales

Swyddogion y Senedd a oedd yn bresennol

Senedd Officials in Attendance

Llinos Madeley Clerc
Clerk
Michael Dauncey Ymchwilydd
Researcher
Phil Boshier Ymchwilydd
Researcher
Sarah Bartlett Dirprwy Glerc
Deputy Clerk
Siân Hughes Ymchwilydd
Researcher

Cofnodir y trafodion yn yr iaith y llefarwyd hwy ynddi yn y pwyllgor. Yn ogystal, cynhwysir trawsgrifiad o’r cyfieithu ar y pryd. Lle mae cyfranwyr wedi darparu cywiriadau i’w tystiolaeth, nodir y rheini yn y trawsgrifiad.

The proceedings are reported in the language in which they were spoken in the committee. In addition, a transcription of the simultaneous interpretation is included. Where contributors have supplied corrections to their evidence, these are noted in the transcript.

Dechreuodd y cyfarfod am 09:30.

The meeting began at 09:30.

1. Cyflwyniad, ymddiheuriadau, dirprwyon a datgan buddiannau
1. Introductions, apologies, substitutions and declarations of interest

Good morning, everyone. Welcome to the Children, Young People and Education Committee. We've received apologies for absence from Siân Gwenllian, and there is no substitution. Can I ask Members if there are any declarations of interest, please? No. Okay. Thank you.

2. Craffu ar Adroddiad Blynyddol Cymwysterau Cymru - 2018-2019
2. Scrutiny of Qualifications Wales Annual Report 2018 - 2019

Moving on, then, to item 2, which is our scrutiny of 'Qualifications Wales Annual Report 2018-2019'. I'm very pleased to welcome Philip Blaker, chief executive of Qualifications Wales, and David Jones, the new chair of Qualifications Wales, to his first meeting as chair. Thank you, both, for attending. We're very much looking forward to hearing what you've got to say. We'll go straight into questions, if that's okay, and I'll just start by asking you about the refocused priorities you've said that you have. You've said that you've refocused your operational priorities. What are the main reasons for that?

Can I just, in opening, thank you very much for the opportunity to come to the committee today? Qualifications Wales is, obviously, still quite a new organisation, just over four years old, and as you've said, Chair, I'm quite new to the job. I've been in this situation many times, but not with this hat on, and it's a bit of a different experience.

But I think the key thing for us as Qualifications Wales is, primarily, we're there as an important regulator, and I believe that the people who've been running QW before my time have done a fantastic job. But also, we're very focused on the reform agenda as well. I think we wouldn't be delivering on the ambitions that the Welsh Government have set out for Qualifications Wales if we weren't being ambitious and building on what's been achieved so far in terms of regulation.

Our approach has been, and continues to be, about partnering. I think it's a strong thing across Wales, but we don't see ourselves as an organisation that works on its own; it's very much an organisation that works with a whole range of partners. And if I can say this, I would like to congratulate the board of governors and the board of QW for the work they've done over the last four years, and particularly thank my predecessor, Ann Evans, for doing a fantastic job as chair, and thank the whole workforce for what they've done.

So, we share the ambitions of this committee for education in Wales, as we do for the Welsh Government as well. What we're looking to do moving forward is to build on the next phase, recognising that we are quite new, and certainly, those who know me, my background is very much in further education involving skills and so on, and I'm keen to make sure that Qualifications Wales builds on the great work that it's been doing in relation to vocational education and apprenticeships alongside the more general qualifications.

I think it's important—and certainly people like Philip are making a good job of it—that QW is far more than an organisation that seems to be about GCSEs, A-levels and schools. I know it's not about that, but it's important that we're seen to be more than that and build on the opportunities that we have within the Act.

In terms of focus, our focus is still very much as we set out in our general qualification and vocational qualification strategy. So, we've got strategic plans that set out the areas of focus. I think what we've done in the way that we develop our operational plans, and in the way that we're reporting in the annual report, is we've divided things up into four categories, basically, around reviewing qualifications, reviewing the qualifications system, and qualifications that lie within it, and really there, looking at the effectiveness of qualifications; reforming them, where we see the need to reform them, and it is very much a case of where we see to reform them. David's already talked and mentioned reform being an important part of our work. There is also a balance between if things are working well, then leave them alone. So, it's not a case of reform for reform's sake. Then regulating: our primary role is around regulating those awarding bodies that are recognised by us. So, regulation, and then our corporate resources. So, like many organisations, we're just trying to define things into four nice, easy, transparent, easy for the general public and for others to see what we're doing, in review, reform, regulate and our resources. So, as much as anything, it's just a way of categorising the work that we do rather than changing what we're doing per se.

Okay. Thank you. And obviously, this report covers your fourth year since you were established. How do you feel the organisation's role has developed since your establishment, and do you feel that it has changed in that time?

Yes. It's interesting, the primary role remains very much as it was, and as it was set out in the Act. We're primarily a regulator of awarding bodies that are delivering non-degree qualifications. What we have done, though, is we've developed our processes, so you'd naturally expect, after four years, for us to have gone through a process of maturing.

So, many of the things that we have been doing in the regulation of awarding bodies, for example—the report talks about things like statements of compliance where we work with awarding bodies to understand how compliant they are against the rules that we set. Those processes have become very mature now, so, very efficient, very effective. We are quite pleased with the way that those things are working.

Also, things like sector reviews, if we're thinking about the reform agenda and reviews, those have become quite—I wouldn't say formulaic, but they've become very efficient. We know what we're doing with them now. One of the reasons why we had a sector-based approach is because we know that each sector will be different, so we're not expecting the outcomes to be the same, but we're expecting to go through very similar processes in trying to understand the needs of a sector. So, we've become much more mature in terms of the way that we're working and the processes that we're using.

We are quite careful, in terms of our role, to preserve two things that are important. One is our independence, so I'm getting that in early, because it's something that's very important to us in terms of being recognised as an independent regulator, so we preserve our independence. And also, making sure that when we work with others, as David's mentioned—so, we do a lot of partnering—we have to play to what our role is. We have a defined role within the education system and we want to make sure that we're helping others wherever we can, but from the position that we occupy within the system, rather than trying to occupy anybody else's position within the system.

09:35

Thank you. Finally, linked to that, really, from me is: what is the balance, then, between your regulatory role and the contribution you might make to policy development in the field of qualifications?

Do you want to start with that, David?

Yes, if I start with that one, I think, if we're honest, it's a balance that we have to have. We've got to get the regulatory side right, and, certainly, there are two Thursdays in August and there are other dates in the year when we've got to get things right for young people in Wales. But if we just focus on them, and I think QW's doing a very good job of that—beyond that, there's so much experience and knowledge in the organisation that we have a part to play in the team Wales approach to education. So, I think, even though we have that independence, and Philip is absolutely right to emphasise that, it is a partnership with the Welsh Government—I'm sure we may get on to talk about the new curriculum review shortly—but it's more than that. It's about the way that we work with Estyn, the way that we work with the Coleg Cymraeg Cenedlaethol, with providers and others.

Certainly, from my relatively short experience of working in Qualifications Wales, I'm learning already that, as a new organisation, some of the ways that Qualifications Wales is going about its business are setting really good standards, compared to perhaps the way things are happening in other parts of the UK, where maybe there are larger organisations that are sometimes a bit too large and, because of that, don't naturally partner. I think we've got a lot to offer as an organisation, and, certainly, after a couple of months, I've just been struck by the professionalism of the organisation. Having worked in an executive role for 35 years at a senior level, you get a feel for a place straight away. When you go to the QW offices in Newport, you get a feeling of an organisation that's about standards and quality and a real, good commitment to want to do things right, to high standards, and we need that in a regulator.

From a policy perspective, there's always, I'd say, a relatively fuzzy boundary between us and Welsh Government in terms of policy boundaries. So, Government retains overall education policy, and if you're thinking about overall education policy, there are certain things within qualification policy that fall into overall education policy as well.

So, if I give a couple of examples. At the moment, at the start of our work on the new curriculum, we're consulting on two things: one, retaining GCSEs as a title for the main qualifications for 16-year-olds; and, secondly, retaining a skills-based qualification rather like the skills challenge certificate. Now, both of those things actually would fall into Welsh Government's boundaries as big education policy issues. We've clearly got an interest in those, we've clearly got a view that we think we want to test through the consultation, which will lead to advice to Government. But, ultimately, that policy decision will lie with Government around whether they want to retain qualifications at 16, whether they want to call them GCSEs, and whether they want to have a skills-based qualification that sits within them.

There are things that we can do in that sort of policy interface, things that we can do within our own boundaries that help with general policy decisions. So, an example there would be something like the review we've done on construction and the reformed qualifications for construction, where we're looking from an FE perspective at having a foundation, a progression and a apprenticeship qualification for construction and the built environment, for building services. In those new qualifications, we're very aware of overall Government policy in terms of apprenticeships. So what we're doing in looking at that foundation qualification, it's a foundation to get people into apprenticeships where they will get real-world experience to develop the trade skills that they need for employment. The progression route is an opportunity to continue if you can't get into apprenticeship, so that you can continue your education and get yourself in a better place to get an apprenticeship. We're very much working within the boundaries or within the policy environment of Welsh Government around apprenticeships being the best place to pick up those trade skills and promoting apprenticeships. So I think there are some boundaries where we would want to advise others in policy decisions, and some areas where we can make policy decisions ourselves, which interface with the overall direction of policy.

09:40

Okay, thank you. We've got some questions now on reform and review of general qualifications from Suzy Davies.

Thank you. There may be a little bit of spillover into vocational, but you'll have other questions on that, so if you can just stick to general, that will be helpful. The curriculum is out next week. You mentioned just now, Mr Blaker, the start of work on the new curriculum, and referred to the consultation. Do you have a clear enough picture at the moment to help you decide what types of qualifications we're going to need on the back of the curriculum?

I think the curriculum will provide us with a basis to progress with our work on qualifications, so we're not seeing a problem with the curriculum from that perspective.

It is clear enough at the moment. I think there are some differences, though, between qualifications and the curriculum. The curriculum will always be described reasonably broadly, and you'd want it to be described reasonably broadly, but when it comes to qualifications at 14 to 16, necessarily they tend to be described more specifically. The reason there is that you have to balance fairness for the individual. So, when a learner sits down in the exam room and they're presented with a paper, and there are a series of questions on there, you would want them to be able to have had every opportunity to learn the material that they're going to be assessed on, and the only way that you can do that is to be reasonably specific about what the requirements of the qualification are.

I think there is a danger there that can lead schools into narrowing the curriculum to just deliver on the requirements of qualifications at 14 to 16. So I think one of the things that hopefully the new environment and the new curriculum will do will be to encourage schools to reconceptualise 14-to-16 education as delivery of the curriculum within which there are the qualifications that are being delivered. Because I think it's a real negative thing to reduce the curriculum to just the requirements of the qualifications. So I think there's a trick there, and it will necessitate a different conceptualisation of 14-to-16 education to really understand that the curriculum doesn't stop at 14 and qualifications start; the curriculum is a curriculum for three to 16-year-olds, and qualifications occupy some of that space in 14 to 16.

I accept what you say there. Part of this is about moving away from teaching to the exam as an idea. Nevertheless, in that 14-to-16 space there we are going to have young people who want to get qualifications in order to be able to compete with the rest of the world, if I can put it like that. What is the risk, do you think, that because there is this emphasis now on this curriculum being three to 16 and qualifications aren't the core of it, that there'll be a temptation to do fewer qualifications in order to create space within the school day, or the FE day, to do non-examinable areas of work?

We've done some research and in the consultation we talk about the average number of qualifications that young people in Wales are taking, which is over 12 qualifications—12 point something qualifications are being taken. Arguably, that is too many, and arguably anything more than 10 probably doesn't add a huge amount of benefit. I personally think that, if a sacrifice were that there were slightly fewer—clearly, not reducing it down to a very small number—if there were slightly fewer qualifications taken and more time for a good general education that prepares people for future requirements, I think that would probably be a good thing.

And actually, that's one of the reasons why we are consulting on retaining a skills qualification and see a real space for that, because in the development of skills those are the sorts of flexible skills that you want people to develop so that, when they move out of learning just around subject disciplines and they move into the world and they're having to apply a broader range of skills, they've got those skills and they have a good understanding of what's required. So, I think that there is scope to reduce the number of qualifications and make the quality of delivery better. I don't think more is necessarily better.

09:45

Well, I agree with you, actually, but the skills challenge certificate itself is part of a bigger qualification at the moment, obviously. Can you tell us a bit more about the work you're currently doing on that and whether there's a risk in there as well of still making it something that produces a piece of paper at the end?

Clearly, we did a review on the skills challenge certificate, and when we looked at it we found a number of issues associated with it. Some of those were quite technical issues. So, looking at the coherence of the assessment model, and actually the amount of assessment that was involved, and also looking at the burden, the manageability of the qualification for providers. Now, we're very much of the view that there is still a space for a skills challenge certificate, but we are consulting on it. We think it's one of those things where schools have become much more familiar with the skills challenge certificate, and they've become much more familiar with the delivery of it.

In fact, as we've been looking at what the redesign of the qualification might be, we've found that there have been very diverse views. There is actually quite a large pool of people saying, 'Leave it alone. We've got used to delivering it now.' The appetite for change varies quite broadly. That said, we do think there's scope to make it more manageable. We do think that there's scope to make it more coherent, and we're going to be looking at that and consulting on it later this year, in September and October. We've got proposals that we want to consult on. 

It's worth noting that—the skills challenge certificate framework, so to speak—there were four qualifications that sat as skills challenge certificates. There was the national foundation qualification for 16-year-olds; the advanced skills challenge certificate for 18-year-olds; and then there were two single-year further education versions at foundation and national level. We've worked with Welsh Government and WJEC to discontinue those two FE qualifications, because they had a very low uptake. What we've ended up with is we've now got a more simplified model of just the 16-year-olds' and the 18-year-olds' version. So, we will consult in September and October for new proposals for those.

I think one of the things we want to do is actually to draw out in those new proposals much more about the development of skills. Because there is a danger that, actually, what's happened is that things have been reduced down to a set of content that's being delivered in schools, and perhaps not being always delivered with the amount of curriculum time that's necessary to develop those skills. It's the sort of thing that's being shoehorned in at the beginning of the day or the end of the day or around other things, rather than a discrete exercise in developing skills. So, I think what we want to do is we want to have a qualification in the future that is much more about developing skills, less about learning particular things, making it less predictable as well, giving it more scope for opportunities. 

Some of the things that we're looking at are, as an example, and these are proposals that we're still working up in our own minds, for the advanced one, looking at broader perspectives internationally and looking at some of the ideas that might be out there in terms of United Nations sustainability goals and how those can be progressed. And then, for the 16-year-olds, we're looking at the well-being goals in the Well-being of Future Generations (Wales) Act 2015 and looking at how schools can be developing tasks, exercises and experiences for young people that are around developing their skills within the context of those domains. 

09:50

That's interesting. The new elements of the curriculum, like well-being, could be captured, potentially, in a certificate. Just briefly, before we move on to my last question, you're going to consult on the skills challenge certificate, but do you have any worries about the status of that as compared to what's currently called GCSEs? Do you think there'll be an issue around that, or is that something for the consultation? 

I don't think so. So, it's interesting. As the committee will know, we've got a higher education engagement officer who goes out and talks to universities. Actually, he's engaged with more than 100 universities now. And we've been compiling a database of offers that universities are making, so we can understand how the advanced skills challenge certificate is being used in offer-making strategies by universities. And actually, it's really positive.

There's really starting to be a good understanding. Certainly, the work that we've been doing in reaching out to HE has allowed us to clarify issues and, where there might have been misunderstandings, to address those misunderstandings. I know that one of the headline concerns is always what do Russell Group universities think about the skills challenge certificate. Many of them are including it in their offer strategies at the moment, so offers are being made against it. Very few universities aren't making offers against the advanced bac skills challenge certificate. And for those universities, what they're saying is it offers candidates real opportunities in terms of being able to exemplify their learning. And they're seeing the benefit of the skills as they're coming into university.

So, I think its value is consolidating at the moment, as it's becoming more established. And that, of course, has got to be one of the things that we balance when we look at changing it. Because we don't want to chuck the baby out with the bath water; we don't want to get into a position where something that is starting to be understood and being valued more and more is changed to a position where it isn't valued. 

Okay. Thank you. Then just briefly, following the consultation that's currently ongoing, can you give us a flavour of how you're going to use that material for developing what we currently call GCSEs or general qualifications?

Yes. What we felt was necessary in this first round of consultation was really to go back to basics. So, we're consulting on the very basics and building on that as we start to move forward with more and more detail. Now, part of that is because we think that there is a need for social consensus, really, on the work before we progress too far, but also thinking about the fact that we wanted to get started with things but the curriculum hasn't been published yet. So, we needed to get on with some work before the curriculum was published, and then think about the next stage for ourselves. 

Can you give us an idea of some timing? Because obviously you've mentioned other consultations.

So, we're seeing a consultation on the skills challenge certificate in September/October. We're also seeing, probably towards the end of November/December, the next round of consultations in relation to the curriculum. That will be the one that will be the big one from schools' perspective. The principal things we'll be looking at there is the design principles for GCSEs. So, what do we think that these new qualifications that we'd like to call GCSEs—or whatever they might be if GCSEs don't get consensus. What do those qualifications look like? What are the parameters going to be around them? And also, what subjects are going to be in the mix?

If we took an example of something like humanities, there is an AoLE on humanities. You could describe a qualification just at the humanities level and have an integrated humanities. You could at the other extreme retain all of the existing subjects and have geography, history, business studies et cetera that sit within that AoLE. Or you could have some options that might exist in the middle.

What we're going to do is we're going to, over the next few months, be working with a group of experts to develop the design principles and develop our thinking on those design principles, so that we've got firm proposals to go out to consult on at the end of the year. Alongside that, we're going to be thinking, 'Okay, if we're developing those design principles, what's the impact on how we describe the domains, how we describe the subjects?' And then thinking about the assessment models that would work best for those subjects, and bringing all of that together in the proposals that we'll consult on at the end of the year. So, that's the big one. 

The reason why that's very important is, if we're thinking about schools, schools will want to start thinking about how they construct their infrastructure ready for delivery of the new curriculum. They'll want to know whether they've got integrated humanities, whether they've got individual subject disciplines and whether there are training issues for teachers associated with moving away from subject disciplines. So, it's a very important stage, and we think that we'll be in a position to consult on that, say, in November.

09:55

Okay. I can have a 'yes' or 'no' to this one: do you think it'll also have impact on ITE?

Okay, thank you. We've got some questions now on vocational qualifications from Dawn Bowden.

Thank you, Chair. Morning, both. You touched earlier on, Phil, about the sector reviews that you are involved in. I think you're probably about halfway through them now. You talked about what you've done so far as developing a more mature approach to them. What have you learned from it, and how is it perhaps changing your approach to the way you review?

I think we've learnt a lot. One the reasons why we wanted to go down the sector approach was we had always assumed that sectors would be different, and that what one might decide is an appropriate approach for one sector might not work in another, and I think we've found that to be the case.

If I give you one particular example—actually, I'll give you a couple of examples. So, for health and social care, qualifications are a really important part of the landscape. They're there as a licence to practice. They're there as an important part of the career progression framework for people in that sector. Similarly, with construction, they're a licence to practice, in effect, where people will get their trade cards so that they can go on site and work in their trades, and they're an important part of the infrastructure. That's why, for both of those sectors, we've done some quite fundamental reforms, where we're looking at changing all of the qualifications that sit within those sectors.

Yet, for the IT sector, which is one that we reported on in December 2018, qualifications were a less important part of the framework. What employers were telling us was that, for the higher-order skills, they were really looking for graduates. Graduates with broad skills, like communication skills and being able to work in teams. Therefore, it wasn't really practical for us to look at changing all of the qualifications there. What we wanted to do was to make the route through to higher education as efficient as possible. What we've done is we've allowed the legacy ICT GCSE and A-level to persist while we continue with the review, and we've now decided that we're—. Well, we're in the middle of developing a new GCSE and A-level for digital technology, and that'll really provide that route through to higher education for those learners.

So, it's a horses-for-courses approach, rather than a one-size-fits-all approach, then?

It is, absolutely. At the moment, we're looking at engineering, advanced manufacturing and energy. We're in the middle of the review phase of that, so we've completed our primary research. We're consolidating that primary research and starting to think about what the implications are for that, and it looks like it's a fairly mixed picture there. The current qualifications are held in high regard. So, we haven't quite decided what we're going to do.

We are about to start what we think will be our last sector review, which will be looking at travel, tourism, hospitality and catering, that sort of area. And the reason I say that's likely to be our last sector review is that sectors need to be of a certain size in order to have agency off the back of them. So, if we're going to spend two years looking at a sector and really getting to understand it, we need to have confidence that, at the end of that, we can have the agency to do something about it. So, we think that's the last sector that will be big enough for us to be able to do that with. And what we'll be looking to do, over the next 18 months, is to develop what our approach is going to be outside of that, and thinking about what we'll be doing with some of the smaller sectors.

Okay. Certainly, from the first three reviews that you undertook, you talked about the need to develop and rationalise the incoherent and disjointed offers that were previously there. So, what lessons have you learned through the commissioning and the development of qualifications? I think you've got City and Guilds, WJEC, City and Guilds/EAL partnership. So, what lessons have you been able to draw from taking that approach?

It's a big exercise. Commissioning qualifications is laborious. We dedicate a lot of our resource to that process. We were the first people to take that sort of approach with health and social care qualifications, all of which aren't yet in place. So, many were introduced for first teaching in September of last year; some more will be introduced between now and September of this year. It's a big exercise.

What we've learnt is—. We've actually changed some our internal structures. So, we've changed internal structures to make our own internal operations more effective. We've learnt that, when we're procuring and going out to commission, with construction, in parallel to going through that commissioning process, we worked up our ideas in a lot more detail than we had previously with health and social care. So, we were much surer about what we were going out to procure and where we wanted to go.

We've also learnt that, when an awarding body submits a qualification to us for approval, an awful lot of work has been done by them and, actually, if it gets presented to us for approval and we don't like it, it becomes a very intensive exercise in reiterations to try and work things through. So, we've introduced a pre-approval process, where draft, outline documents get presented to us, so that we can check whether the awarding body's heading in the right direction before they've gone into all the detailed processes.

I think the other thing that we've learnt from health and social care is that it's not just about developing the qualifications. A lot of it's around the change management process and working with Government on changing apprenticeship frameworks, making sure that the funding's in place for further education. So, there are lots of change activities that need to go on around the outside of developing these new qualifications, and we've started—. We're really attuned with that now, and putting an awful lot more work into that change management activity.

10:00

Okay. That's helpful. Thank you for that.

Can I move you on now to your review of Essential Skills Wales? Now, you might have been aware that some stakeholders had said that the approach, the consortium approach, to developing the qualifications had caused issues. They talked about the four disparate awarding bodies, with different processes and different ways of working. You might well be aware of that. So, how do you think that building the partnership with City and Guilds and EAL for developing new building and construction qualifications is going to avoid those kinds of issues in the future, if at all?

So, I think they're two different things. So, in essence, with essential skills—. So, the essential skills model of awarding bodies working together was developed by Welsh Government before we came into existence. So, it's something that we've inherited. There are contractual arrangements in place for those qualifications. So, it is a consortium in a looser sense, in terms of its four awarding bodies working together to try and develop qualifications and sharing resources, and, importantly, sharing Welsh-language resources. Now, we've introduced an oversight board, which brings together senior people from those awarding bodies, so that they can work together at a senior level as well as advising a group that sits below, on a more operational level.

It's not a body—it's a governance arrangement, which sits over the top of it. That seems to be very effective, and, actually, from an essential skills perspective, we're getting very few concerns raised with us now. Actually, what we're thinking about doing at the moment is extending the life of those qualifications by a further two years, because we think a lot of the issues that were there to begin with, associated with new qualifications, have become resolved. There are more control tasks that are available that are contextualised. So, a lot of those issues have been worked through.

With the consortium for construction and built environment between City and Guilds and EAL, in effect, that is City and Guilds acting as a prime contractor for us, with an arrangement with EAL—much more formal, much more contractual, and we don't anticipate that there will be any of the issues that we see of a looser arrangement of four awarding bodies working together, rather than a proper consortium approach.

Okay. That's fine. Thank you. I don't know whether your voice is going to go before mine, but—.

Can I just move you on now to the delivering digital review? And that showed that many of the qualifications were out of date. So, with the sector review process looking like it will take around a decade to fully complete, what's been put in place to ensure that awarding bodies will be able to keep their qualifications up to date during that period?

10:05

So, I've already mentioned a new GCSE and A-level to replace ICT, which are actually probably the highest uptake qualifications anyway. So, for those highest-uptake qualifications, we're doing something. One of the requirements that we have in our approval criteria for those new qualifications is an obligation to keep the content under regular review. What we're working on with WJEC at the moment is to find out what's a pragmatic view of how frequently they should be updated, because changing content very rapidly can actually be unsettling to schools and they never quite understand what they're delivering. But, equally, we don't want to be in a position where—. I think it was only a couple of years ago that 3.5 inch floppy disk drives—

—came out of the ICT qualifications, and I think the only place you could find a floppy disk was probably in the GCSE. [Laughter.] So, there is clearly an issue, and, for those qualifications that are higher uptake, we're clearly doing something—for the GCSE and the A-level.

What we are also doing is—. Qualifications regularly come up for renewal. So, we will designate qualifications, and, when we designate a qualification, it has an end date. When it has an end date, it can be resubmitted to us. When it's being resubmitted to us for ICT qualifications, we're having an additional check to make sure that content has been updated. So, as qualifications are regularly going through the cycle of being resubmitted to us, we're making sure that content's up to date.

And that's probably more important, isn't it, in digital, than in many other subjects, because it's such a quickly moving, developing area, isn't it?

Yes. And I think it's important also to remember that one of the things around IT is that many of the qualifications that are used by employers will be ones that sit outside of the regulated framework. So, they'll tend to be vendor-type qualifications, and those vendor qualifications will relate to specific pieces of IT or software packages or development packages that they're offering. So, they sit outside of our area of regulation and our oversight. Those things change very regularly anyway, because every time they release a new version of something, there's a new training package and a new qualification that comes along with it.

That's fine. Thank you. So, in your vocational strategy, you already acknowledge that Wales is a small market compared to England, so what risk does that give rise to? Or maybe it doesn't—I mean, have any materialised? And, if they have, what are you doing to mitigate those potential risks?

So, I'll just try and come in on this one, and give Phil a bit of a rest. I'm sure he could lend me some detail, anyway. But it is a risk, and, of course, because of the scale of the English market compared to Wales, it means that the English market dominates UK qualifications. So, that is a challenge.

At the moment, there are two areas of particular concern that we are dealing with. One is the change in apprenticeship assessment in England, particularly on the move towards end-point assessment, EPA, which is different to the approach that's being maintained in Wales, which is involving the qualifications that exist at the moment. So, I think that is a particular challenge. We—and I certainly personally believe that the Welsh Government—need to have a really close eye on developments in apprenticeships, because, again, as somebody who up until fairly recently worked at a college that's right on the border with England in north-east Wales, that transportability of apprenticeship qualifications across the border for many companies who have a workforce across—it happens to a degree in mid Wales and to a degree in south-east Wales—is a challenge, and I think there is a bit of work. With the primary importance of apprenticeships—more and more of them, more important for the economy and for the opportunities and the choices we give young people—I think we're going to get to a position very quickly where there needs to be a review of the approach to apprenticeships in Wales, and I think Qualifications Wales has an important part to play in that.

Clearly, on the horizon, there is the development, following the Hazelkorn review, for the PCET reforms, to develop the Commission for Tertiary Education and Research, which, I think, is the current working name of that organisation. I think one of the issues for us is—. I think that's going to be in place about 2022. Well, that's two or three years away from now. In the meantime, the world moves on, and I think there will need to be—. Well, I actually think the new body, CTER, can learn from the development of Qualifications Wales, which is a body created from the merging of different institutions. But also I think it's important, back to Qualifications Wales's partnering approach, that—and Philip is already part of a working group for that—we get involved with them early on, because I think there are going to be some grey areas, some overlapping areas, between the work of Qualifications Wales and CTER and possibly others as well. Who is responsible for making sure apprenticeships work in Wales? We know in other parts of the UK there are some turf wars going on, which aren't in the interest of employers and learners. Whereas I think, in Wales, the approach we have to take is that we have a collective responsibility, rather than somebody dominating. So, I think that's a key one, so that goes back to apprenticeships. And then the second one that I referred to is the development in England of T-levels, and they're a bit Marmite, they are. Certainly, from ex-colleagues who work in colleges across England, some seem to love them, but, generally, most people are quite negative about them. But, again, we need to be aware of it, because there's an impact on the qualifications that awarding bodies—the large ones that are operating across the UK and internationally—are developing, as well as the impact on learners, who in many cases travel across the border to do full-time FE courses, and so on. So, I think those the two main areas. Philip, anything you'd like to add?

10:10

I think also, with sector reviews—. So, with sector reviews, where we go and develop qualifications for Wales, we sort of isolate ourselves from those influences and policy in England. So, for health and social care, for construction, as examples, those changes won't affect us, because we've got our own qualifications.

Now, actually, that's one of the considerations that we have, going forward, with engineering. So, if we're looking at engineering, advanced manufacturing and energy as a sector area, there may not be a driver in itself to change the qualifications, because they're effective as they are at the moment. What we may want to do—or what we will be doing—is risk assessing what is the sustainability of those qualifications, given the UK context, and then understanding whether we need to go ahead and commission something, not because the current qualifications aren't sufficiently good, but because we can see a long-term risk to the availability of them in Wales. So, there will be some work that we can—

Yes, and what we've also done is we've developed some very good relationships with awarding bodies. So, particularly on apprenticeship frameworks, where there's a qualification that we think that might be at risk, we're working to talk to the awarding bodies that are delivering those qualifications to understand what their plans are. And, actually, there are some opportunities. So, even though, in the UK context, England policy dominates what might be available, those big awarding bodies tend to have an international offering, which is less dominated by policy in England, and, actually, what we're doing is we're sort of looking at whether some of those international offerings might be offered in Wales. Indeed, one awarding body has recently launched a new international qualification in the hospitality and catering area, and they've run a session, I think this week, in Cardiff to launch it in Wales. So, we're quite positive about that, because there are some very good qualifications that are being made available that way.

Okay, that's helpful. Thank you. My last question, Chair, is just about—. You've talked quite a bit, actually, about health and social care and building and construction skills, and City and Guilds have been commissioned to develop qualifications in your first two reviews. Do you recognise the risks around awarding just to one body, and whether they become almost a monopoly in terms of the delivery of those qualifications?

Clearly, the process around restricting qualifications is a mechanism that we use to make the development of something new and different viable for Wales. I think the issue is around, when we go out and do an open procurement process, do we want to make sure that we're getting the best awarding body to be able to deliver that qualification? And are we going through a procurement process that is open, transparent, and relates to clear criteria? That's definitely the approach that we've got, because we want to have compliant procurement and make sure that we're doing the right thing. And what we have there is City and Guilds has been successful twice against often—you know, having the best offering. The context is slightly different, so, to begin with, it was City and Guilds and WJEC that both had different parts of health and social care, and what they've done is they've worked together on those, and what we've got, with construction and the built environment, is City and Guilds with EAL working with them. Now, there are some risks there; yes, we're alert to those risks. There are also some advantages. So, if we're thinking about developing Welsh language capability, they've got scalability in being able to develop some of those capabilities, which means that they're able to be more responsive. And also, they're able to understand what the support and resources requirements might be in Wales, which will be different to the support and resources requirements that might be in place in England, where, in Wales, it's a smaller market, so it's less commercially interesting for commercial publishers and the like. So, there's a recognition of the resources that are needed. So, there are risks but there are real advantages as well.

10:15

Yes. Just on this question of Welsh language, and going back to the other question about your influence on policy when you're working with Welsh Government. We're familiar with complaints that, even though qualifications are able to be delivered bilingually, they're not capable of being assessed bilingually. So, does that have an effect on your choice of which sectors you prioritise? So, for example, if something is a very heavy Welsh language—say agriculture as an example,—you wouldn't necessarily prioritise that because it could be an incentive for people not to bid to offer qualifications? And if there is an issue with the number of assessors who are able to assess sectors through the medium of Welsh, is that something you've taken to Welsh Government, who may, of course, be able, then, through, I don't know, IT or other sources, to raise the profile of the Welsh language assessment capacity?

What we've started doing, we have a forum for awarding bodies that are interested in delivering qualifications through the medium of Welsh so that we can get them to share best practice in identifying—. Because they need internal verifiers within schools, external verifiers, markers, whatever it may be, there is an element of externality that they need to be able to source.

So, we've got a self-help group, which we facilitate, with awarding bodies coming together. The other thing is, Coleg Cymraeg Cenedlaethol started to take responsibilities in FE. So, if we're looking at the qualifications we're commissioning—so, for health and social care, and construction—there is a clear requirement for them to be delivered bilingually because they're approved qualifications.

Yes. And those awarding bodies are working hard to recruit the markers that they need. There is a shortage. It is a constraint. I think that we, collectively, not necessarily just us as the regulator, need to do more to try and promote an interest in taking part in qualifications actively. So, getting more people there. I think the Coleg Cenedlaethol can really help with that. I think that there's a real opportunity to do more there. We've been looking at ideas like whether we might have some sort of central database and start to be able to get something. We need to be cautious there, because as a regulator, we don't want to necessarily become trapped into it—you know, where an awarding body that might have a difficulty delivering something, 'Well, it's the guy that you put us in contact with that's the problem.' So, we need to be careful about how far we go, but we're actively looking for areas where we can help to promote the recruitment.

Could I just add really quickly to your question? I think, also, that the Coleg Cymraeg Cenedlaethol, as Philip refers to, and the responsibilities they now have in relation to FE, apprenticeships and so on, they've developed groups, and I think they are addressing many of the concerns or issues that you've raised there, because they are genuine concerns, and it's about people working together.

And the other thing I would add, from a Qualifications Wales point of view, we are in the process of developing a new strategy for Welsh language and bilingual provision, and that will be completed by the end of the first quarter of 2020. And that recognises the priorities that we have in terms of the Welsh language, recognising there'll be separate issues linked to the new curriculum, and the issues around, perhaps, the Welsh language within that.

Thank you, Chair. What are the main changes to your conditions of recognition for awarding bodies that you consulted on recently?

Yes. We've worked closely with the other regulators. We undertook a review of the conditions over the last couple of years, working with awarding bodies. One of the clear things that they told us was that it's useful and helpful and reduces their burden to have a common set of requirements across the UK regulators. So, we led on a review and have now worked, together with the Office of the Qualifications and Examinations Regulator and the Council for Curriculum, Examinations and Assessment Regulation, to look at the rule book across the three countries.

The main areas where there have been changes have been around fee information, so, placing a much stronger requirement on awarding bodies to make their fees clear, to make sure that if there's any packaging around the qualifications, those fees are clear, any additional costs that might be there, and also, that they're publicly available. So, in other words, when a provider or a learner is making a choice of qualification, they've got an absolute understanding of what the offering is. Sometimes, that information wasn't available publicly and then there's a danger that there are deals being done and it's not equitable. So, we wanted to make sure that there was clear fee information.

The other area was around issuing results, so we've introduced a new condition where we can stop an awarding body issuing results. That would be done in extremis, so that would be where we felt that there was a problem that was of such significance that results shouldn't be issued. Now, we could issue a direction to do that anyway, but we've introduced a condition on it so that if we need to, we can. That would be the sort of thing that the three regulators would probably work on in conjunction with each other to introduce.

The other is making it clear about processes for recognition of prior learning, so more conditions around that. Essentially, a lot of it has been making it more transparent and easier for awarding bodies to understand what compliance looks like. So, one of the things that conditions necessarily end up as is—they're quasi-legal documents, so they end up being in quasi-legal language, and quasi-legal language can quite often be open to different interpretations by the lay, and most responsible officers will be the lay. So, what we've been trying to do is to try and introduce a level of transparency. They're still quasi-legal documents, so they still need to be described in those sorts of ways, but we've been trying to move to make them more transparent.

One of the things we'll be doing in the future is more guidance around it and working with the other regulators to have a common set of guidance so that some of that extra transparency can be introduced in terms of guidance that sits alongside the conditions.

10:20

Thank you. What were the reasons why the three applications from organisations to be recognised awarding bodies in the last year did not meet the criteria for recognition?

Each will be considered on its own merits. The criteria relate to whether we think that the organisation will subsequently be able to be compliant with the conditions. The sorts of things that the three awarding bodies didn't get through for recognition on would have been over things like governance arrangements, which weren't sufficiently clear or transparent, conflicts of interest and capability to develop and deliver qualifications. So, essentially, it's where we didn't feel that they would meet the requirements.

What we've done is, whenever we give an outcome on that, we provide detailed reasons why they weren't considered to meet the criteria, and then an awarding body can resubmit to us. So, one awarding body has resubmitted, and we understand that another one intends to. So, we're very transparent around the reasons why they didn't meet the criteria.

Okay. What progress are the five recognised awarding bodies that were not in compliance with the conditions of recognition making to reach full compliance, and how likely is it that they will lose their recognised status?

So, out of the five, one has surrendered its recognition anyway. So, one awarding body is no longer recognised by us. The other four are relatively minor areas of non-compliance, so if we felt that they were significant areas of compliance, we would have taken stronger action with them; we would have issued directions or the like. So, we would have been more active with them. They all have action plans in place, so we're monitoring those action plans to make sure they come back into compliance, and we're getting a check on them in February, so we'll have an understanding of where they are.

But to give you an idea, it's things like one of the awarding bodies has some digital transformation going on internally and, as a consequence of that, they haven't been able to publish their specifications in a way that they would want to and would be compliant. So, they're doing other things to mitigate that. So, they're not major areas of non-compliance.

Okay. What is the relationship between the data on incidents of malpractice notified to Qualifications Wales by awarding bodies, which is included in your annual report, and the data on numbers of penalties issued for malpractice in your separate statistical release? When viewed in combination, what do they actually tell us about the level of malpractice in recent years?

10:25

There are lots of overlapping data sets here with slightly different time frames, and covering different things, so they can be quite confusing. The 23 incidents of malpractice or incidents that could have a significant impact on learners, or have the potential to, are actually maybe not proven cases of malpractice—they're incidents that could have an impact on learners. Now, out of those, a number of those have actually manifested themselves as malpractice cases, proven malpractice cases.

So, something about incidents that could have an impact on learners, and then, the statistical release that we have is actually those cases that have been proven. In those, in the last release for last summer, which said that there were 200 cases of malpractice, most of those were individual learners and most of those were mobile phones. So, the most common thing is that, like I have here, you have your mobile phone in your pocket. Young people particularly feel that they are biologically attached to their mobile phone. Indeed, I was on the malpractice commission that the Joint Council for Qualifications had, and one thing that there was quite a strong argument for there was that there was such an emotional attachment to mobile phones, should there be a way of making sure that they're switched off and put in a bag on a table? Because young people were having anxiety associated with being disconnected from their phones.

The classic one, which does happen more often than one might think, is that somebody has inadvertently got their mobile phone in their pocket, and at the end of the exam it's their mum who phones them to see how the exam went, and they're still in the exam room, and that proves the fact that they had the mobile phone on them. So, those sorts of things can happen. That's the main thing. We don't see there being any—. Clearly we're concerned about malpractice, clearly we're concerned about any centre malpractice, and those cases of malpractice tend to be around—the most common area is too much help in non-exam assessment, so, practicals or the like. So we're clearly very cautious about that, especially as we look forward to developing new qualifications for the new curriculum. If we want to have more along those lines, of non-exam assessment, it's really making sure that they can be delivered securely, and a lot of that comes down to things like performance measures and accountability, and making sure that schools sit within the right environment.

I'll do them backwards, if that's okay. Designated and approved qualifications—most of the work on approved qualifications has been with the new qualifications, hasn't it? Is this status of designated qualification in your view just an interim position that you've got to work through over time, or is it a permanent situation with all your work on approval going on what's following the curriculum, which is obviously going to take a lot of time?

Realistically, it's a permanent situation. We don't see it as being problematic, either. So, over time, we've increased the number of approved qualifications and we've reduced the number of designated qualifications. I think there will be a continuing trend in that direction, where we'll look to reduce the number of designated qualifications. As an example, with the curriculum, we said that there are nearly 2,000 qualifications that are designated for use by schools for 16-year-olds. We would, through the application of what we're proposing in the consultation, see that number come down quite considerably. But it's not problematic. We are looking at—. At the moment, we approve things where we're developing something new, and they have very detailed approval criteria. As part of the work looking at the new curriculum, we're thinking about whether there might be broader approval criteria that get applied, which might mean that it's a different process for qualifications coming through. But I think there will always be designated qualifications, and they are principally where it's a UK offer that is being regulated by Ofqual, CCEA and us, and it defines that territory neatly.

And what about those situations—I'll just use an example—obviously, it may not come to this, but if you end up going for a more generalised humanities GCSE rather than specific subject areas, would it be the case then that, I don't know, history and geography and business studies would fall out of the designated category and disappear altogether because you'd have a new, approved humanities—?

They're all approved anyway. So nearly all GCSEs, or all of the high-uptake GCSEs, are already approved. So the only GCSEs that are designated at the moment are where there are very low uptake qualifications, where it's not viable to have them developed for Wales. It's actually one of the reasons why we see there probably still being a space for designation, moving forward, because even when we looked to have a more defined position for schools, we wouldn't want to be in a position where, let's say, for Ancient Greek as an example, you might only have three or four learners, but would you want to be in a position where they couldn't take them? Probably not. So you'd want to have a designated space to allow access to those qualifications, even though we wouldn't want to be in the position of approving them with such a small number of learners. 

10:30

Okay. Thank you very much for that. 

The Welsh language continuum: do you want to give us a little bit of a steer on where you think you might be going with this, perhaps with a single qualification, accepting this is a bit of a journey in itself?

It is. I'm glad you've raised that issue as well. Certainly, following Sioned Davies's report, which goes back quite some time now, last summer, with the first cohort who sat the Welsh second language for the first time, and we developed the provision there with a working group of practitioners in the sector. So that's gone well. Clearly, there are differences of opinion in terms of where that continuum for the Welsh language goes. Of course, with there now being a new curriculum on the horizon, we're looking to the Welsh Government for some steers in terms of the way they want us to develop the Welsh language within the curriculum and then, with it, the qualifications that follow. But certainly what I would say more generally as the new chair and as a Welsh speaker as well is that I think that Qualifications Wales has scope to develop further its Welsh language offer, and I think the new curriculum gives us a great opportunity to make sure that, as part of the bigger Cymraeg 2050 ambitions, we make sure that qualifications are at the heart of achieving that.

As I mentioned earlier on, the new strategy for Welsh language and bilingual provision is an important part of Qualifications Wales setting out a clear plan as to how it's going to do things itself but also how it's going to support awarding bodies and others to make sure that we are able to get that parity in terms of the availability of qualifications, not just obviously the Welsh language qualifications, but other qualifications through the medium of Welsh as well as English. 

Also, Suzy, I don't think that necessarily a continuum means a single qualification, because—

On a continuum, there'll be a broad range of ability of learners, and a single qualification stands in danger of being too easy for those who are very good and are first language speakers and too hard for those from English-medium schools who do not have a Welsh-speaking background. We haven't developed firm proposals on this. We're open and it's going to be an area of work over the coming year, but I don't think it necessarily means there'll be a single qualification. Personally, I think it probably means that there'll be more than one qualification that sits in that space. 

Okay. Can I just develop that slightly, if I've got time? Obviously, if the strategy works and you have children starting school and being raised more bilingually than they are at the moment, the chances are that those from even English-medium backgrounds will have greater Welsh language skills than they currently have at a particular age. Does that mean that, potentially, having more than one qualification could be an interim thing, even if that interim is a long period?

Definitely, and certainly that's where we're looking for more clarity from Welsh Government on what its intentions are, because the curriculum suggests that there's going to be a phased approach over a period of time, as capability builds. So we'll need to understand what qualification offer can best sit alongside those proposals and wider plans. I think you're absolutely right that we may see, over a period of time, the range of qualifications changing, but the one thing that's clear is we will move away from the notion of a second language qualification. 

Okay. That's helpful. Thank you. Just finally, and quite quickly, the last time you were sitting here we were having a bit of a set-to about the grade boundaries up in north Wales. Can you just give us a little bit of an idea of how confident you are about those for 2019, and even 2020, if it's not too early to say? How are you going to monitor the processes for making sure the grade boundaries are fair?

Certainly, from 2019, we monitored nearly all the awarding meetings. We have a very clear role. WJEC make the awards. They have an awarding committee for each subject, which makes the awards. We go along as a regulator to monitor, to make sure it's compliant and that that exercise is conducted appropriately. And from all of those awarding meetings that we've monitored, we're confident that they have been compliant and they've been undertaken properly.

Wales is slightly different from England in terms of how those meetings work in Wales. We have a combination of statistical evidence and judgmental evidence that comes together. In England, there's a stronger weight on statistical evidence because they have key stage 2 tests at age 11, which provide a statistical basis, a stronger statistical basis. But we're very confident that the model in Wales is appropriate. We're confident that it works well. There aren't any significant changes that we see for 2020. So, it's not something that is a terribly mobile thing. It's not dynamic, it doesn't change often, and we're confident that it will be okay for 2020.

10:35

No, not around grade boundaries at all.

Thank you. Well, we've run out of time. We have got a few questions that we didn't cover, so if it's okay, we'll write to you for answers to those. Can I thank you both for attending and for giving such comprehensive answers to the committee's questions? We will send you a transcript to check for accuracy, as usual, but thank you, both, very much again for coming. The committee will break until 10:45.

Gohiriwyd y cyfarfod rhwng 10:36 a 10:45.

The meeting adjourned between 10:36 and 10:45.

10:45
3. Addysg heblaw yn yr ysgol - Sesiwn dystiolaeth 1
3. Education Otherwise than at School - evidence session 1

Can I welcome Members back to the committee for item 3, which is our first oral evidence session for our inquiry on education otherwise than at school. I'm very pleased to welcome someone who used to be a regular visitor to this committee, Ann Keane, who is former chair of the Welsh Government education otherwise than at school task and finish group, which ceased to exist in 2017; and Professor Brett Pugh, chair of the Welsh Government delivery group for EOTAS. Thank you both for coming and we're really looking forward both to the inquiry and hearing what you've got to say. If I can start by asking you if you could just give us a little bit more information about how both the task and finish group and the delivery group have worked. 

I'll start since Brett comes after my time, as it were. The task and finish group was set up in September 2015 and it carried on its work in half-day meetings every two months until March 2017. We had representation from pupil referral units, EOTAS, local authorities, the Children's Commissioner for Wales's office, Estyn and, of course, civil servants. I also had meetings with the Minister to report on progress, met with civil servants to set agendas, and so on. We also had the annual sector one-day conferences and we discussed the progress on the group in those.

And actually, the starting points for the task and finish group were the publication of three reports. One was the University of Edinburgh's very thorough, comprehensive evaluation of EOTAS, published in 2012, I think. Then there was in 2014 a report on pupil referral units by the previous children's commissioner—quite a critical report. And then, there was a survey report by Estyn 2015 and another in 2016 that gave us quite a lot of evidence to discuss. 

So, initially, what we started doing was to corral the recommendations from all those reports into a structure that I put together that identified the issues and priorities. And then, once we started our discussions, new issues emerged in some of those discussions and that led eventually to the publication of the framework for action in 2017.

That's where I then came in in 2017. We took the draft framework for action and formed what was then called an implementation group out of many of the same members that had been on the task and finish group, because of the area of expertise, but included some additional school, PRU and local authority representation there. The group initially met to look at the structure of the framework. We then took it out through the consultation process, finishing then in December with the actual publication of the framework as the baseline for our operations.

We met then on a quarterly basis, and the initial take that we had on this was that we were going to be very much hands-on in terms of the implementation, because the first part of this work needed to really increase the evidence base that we had. We felt that the expertise of the people around the table would be very well placed to be looking at commissioning reports, commissioning evidence, to be working alongside that process and to be actively involved.

And in order to make the best use of time for those people, we actually worked in three sub-groups at that time. The sub-groups changed in terms of their functions. They started off looking at things such as registration, data and then looking at documentation for contradictions, mixed messages. They moved their function on, and that had been the role up until, you will see in my report to the committee, the change of name to an advisory function.  

Through the initial stage, there were a number of reports brought to us, a lot of information, and some of that is still coming forth now, and will be published in the near future. Where we changed our role was when we felt that perhaps the hands on, sleeves-rolled-up approach was perhaps not the best approach and the best use of the expertise from this point forward.

Once we'd actually got some of the publications that will be coming out—the last of them probably in the spring this year, some Estyn ones and some other work on commissioning, very much for EOTAS provision—we thought that what is really important at this point now is to look at the join up of EOTAS provision with the wider reform agenda in Wales, so we don't have that danger of bolt on, and that this is actually seen as a Brighton-rock model, if you like, throughout this process. Therefore, it didn't in any way dilute the work that we were doing, it refocused it away from that immediate hands on into a much more advisory role in what could be the unintended outcomes of some of this work.

So, essentially, we are working now, again, still on a quarterly basis. We are meeting to look at and to provide advice, but we also vigorously question the civil servants about the implementation of the framework, rather than getting so directly involved in it ourselves. I think, because that initial evidence gathering is now over, or will be over soon, we needed to be looking at, if you like, a much higher strategic level. So, that's where the work has come to.

10:50

Thank you. You've both referred to the representation on the groups. Can I just ask: did that include independent providers?

No. We did consider it, but we thought, because some of the independent providers would be in a commercial situation, the difficulty that would place them in. They might be privy to information that could be commercially sensitive, and we didn't want to get involved in that.

The identification of private providers really came out in the 2016 Estyn report. And of course, by that time, we were already going, so we didn't commission any more and didn't ask anyone to join.

We do engage with them, though, in the annual conference. They attend and they spend quite a lot of time talking to myself and to the civil servants.

Thank you. If I can ask Ann, then, in terms of the work of the group, did it cover all the areas that you wanted it to, and were there any areas that you felt received less attention than they should?

We were pretty free to range as widely as we liked, and if you look at the reports that were the starting points for our discussions, you'll see they covered a wide range. Also, just before I joined, there had been a sector conference where they had identified six key areas for improvement, and these were: leadership, accountability, resources, structures, learner well-being, and outcomes. That allowed a lot of leeway in terms of discussions.

Okay, thank you. And if I could ask Professor Pugh, the name has changed now from delivery group to advisory group, should we be concerned that that represents a diminishing of involvement in implementing the framework? Are you still going to be able to drive the officials on that?

Yes. I think, if anything, what we will be doing is more of the driving of the officials. Whereas before, we were actually getting involved with the officials, if you like, on the ground level. We need now, at this point, to move slightly back and to hold to account, if you like, the officials and to give advice about the join up and the huge potential that there is. I think one of the things that I can't stress enough to the committee is the huge potential with the current reform in Wales to be able to include this group of young people. And to see them almost as a transitionary group; they're not always going to be in this place of EOTAS. EOTAS is part of a service. It's not like a school would be, where you go from the age of three to 16 or 18 or whatever; it's a needs-must process. I think that's very important for the committee at the moment, that we look to say, 'In terms of the implementation of the new curriculum, the development needs of staff within EOTAS—', and that includes the private providers as well, making sure they have access to the right kind of training, and making sure that we bang on the right doors to get there. 

10:55

And just in terms of that accountability, say the officials weren't making as much progress as you thought they should be, how would you rectify that? Would you have access to the Minister? How would that work?

I go to the Minister on a termly basis and will feed back to her on the issues that are arising, and she'll question me around them—rightly so. Also, I meet, as Ann was doing, with the task and finish group; I meet with the officials on a one-to-one basis quite frequently, and I will follow things up there and then report back to the committee when we meet as well. So, there have been some quite rigorous points. Obviously, I wouldn't go into the detail here. But there have been things where I've said, 'I feel things need to go a bit faster on this.'

The danger—there's always a danger that we can go too fast and the unintended consequences are then found at leisure, if you like. But we don't want to go slowly, either; it's that middle ground, and that's always got to be—. You're testing it, all the time. My inclination would be to want things to go as quickly as possible for this group of vulnerable young people, but I can also see some of the issues that have been there in the past.

Yes, absolutely. I agree about the need to take a holistic view of the sector in the context of the reforms, but I'd like to make another point here, which is that the chairs of groups such as these are not line managers in the civil service. 

I think that's important for those of us who have been line managers in previous existences in the civil service. 

Okay, thank you. That's very helpful. I've got some questions now on the framework for action, starting with Suzy.

Yes, I want to ask you a couple of questions on the management committees of the pupil referral units, particularly on this issue of accountability. However much headway you're making with officials, if the management committee isn't working at local level, then the effect on learner outcome is going to be pretty significant. Can you explain that relationship between the work of the management committees in PRUs and learner outcome? What issues have you identified as part of the task and finish work that need addressing, and whether you've seen some of that work already being addressed?

This was one of the first commissioned reports that actually turned up in draft before the end of my period, so I was involved in discussing that. So, I can answer that one. Of course, there was a lot of evidence from Estyn reports. If you look at my annual report when I was chief inspector in 2015, there's a large section on pupil referral units and some of the weaknesses in that sector by comparison with the special schools sector.

So, management committees have a role that's similar to a school governing body, in that they should act as a sounding board, offer support and challenge. The issues that were raised in the commissioned review of their function, and in Estyn reports, basically had to do with: the fact that they were too operational, not strategic enough; there was too little training available for members; they didn't fall into the Governors Wales training pattern; they were sometimes too dominated by local authority members, and of course PRUs are animals of local authorities, and so they didn't have that independence that school governors have, necessarily; they didn't use data well enough to inform self-evaluation and planning; basically, they were offering too much support and not enough challenge; and they themselves were not being supported enough, say in the school improvement delivery by consortia, I think a lot of the people in the task and finish group were not totally happy with the kind of expertise of the support that was coming from consortia to EOTAS. 

Now, a management committee handbook was subsequently published in 2018, and I have had a look at that, and I think it does lots to distinguish the functions of the management committees and to give support and guidance to the members. How far that's been influential, I can't say.

This was a transition between the two of us. The need for the handbook was really quite key. One of the things as well that the WLGA report into the role of these committees talked about was the relative inexperience of the people on these committees. When it actually came out in 2017, it looked at those that had been on the committees since 2015 and it found about 64 per cent of them were newcomers. That in itself becomes an issue, if you like. So, the need to have that handbook, the need to put it in the same position as general governor training, if you like, governor development, was very important. I can't stress that enough.

I think that goes on, if you look to the future, because at the moment we're looking very much at developing systematic tools in Wales for self-evaluation within schools, and, and ahe national evaluation and improvement resource, which is being piloted by 22 schools at the moment. However, I think it should be piloted—and this is something I've been pushing for—by PRUs and EOTAS provision as well, and I think that's got to be something really important.

I think these management committees have got to have status. They're not something that is just bolted on to the wider system. They are different to a school system, because a PRU is different to a school system. It doesn't attract a set number of pupils, and they're not going to be there—or they shouldn't be there—for their whole school career. But it really does need to have that development, and also, I would argue, a structure for the staff as well in the PRUs—professional career development for those staff. Because they are, if you like, possibly a cinderella group, in a way.

But certainly, we've discovered the need to engage this group, and certainly the handbook has been well received in terms of laying out—. It's quite a lengthy book, but it's actually quite user friendly, and it talks about the difference between the operational role and the danger of being over supportive in a school and not challenging and not being strategic. And that's the key development point there. 

11:00

Okay. Can I ask you, then: the handbook, to be fair, is only two years old, but there will have been Estyn inspections of some PRUs and EOTAS provision since the publication of that, are you aware that any of those reports have noted the effectiveness of the handbook, perhaps in raising the game of management committees?

What we do have, actually, which I've not been able to see yet—but it is due to be published, it is one of those last pieces of information, early this year, hopefully by February—is that Estyn have actually done a thematic report on the effectiveness of it. 

We're hoping it will be in February, because we want to use it to inform things like our guidance on registration, referral, et cetera, and commissioning. 

So, we'll hopefully get all these in order, if we can. 

Thank you, and thank you, Suzy. We've got some questions now from Dawn. 

Thank you, Chair. Can I ask a quick question to you, Ann Keane, if I might? To what extent did you find the task and finish group identified a lack of robustness in referral processes? Was that something that caused you concern? What did you find?

Among the key themes that came out from our discussions, based on the evidence and the experience of the people around the table, were: referral processes, registration processes, use of data and the reliability of data, and accountability. And those were really quite strong themes in the framework when it eventually appeared.

But certainly, there was pretty much agreement in the task and finish group that processes were not standardised, that with referral there was a degree of ad hocery, that sometimes people would say that it was more about satisfying immediate bureaucratic needs and solving problems rather than considering the needs of individual pupils.

Few local authorities at that stage—I can't speak for now—had multi-agency referral panels that were given a full set of information about the needs of pupils, including from other agencies like child services, care services, youth offending teams, and so on. And sometimes, decisions were made on too little information.

The other side, of course, is: where do they end up? Where are they referred to? And of course, that was another theme: the need for some sort of accreditation of any kind of setting that's used for alternative education, as it were, and that settings should have quality required. 

But getting back to the referral panels, we felt very strongly that the system should be standardised across Wales, it shouldn't be a postcode lottery, there should be multi-agency referral panels, and they should have a full set of information about individual pupils, and about the potential destinations, and that those destinations should be the best ones for the child.

Anther thing that came up in that discussion about referral was managing moves, and we were unhappy about the way in which managed moves were undertaken behind closed doors, that there wasn't the formality in the process that pupils referred to PRUs had. Also, we were worried that there were no exit or entry criteria in PRUs, and terms of reference were not standard, nor even available in some cases, because of the rather arbitrary nature of some of them.

11:05

So, does that mean that there weren't outcome assessments? Did you have concern about the potential outcome for the students placed in any of these units? Was that one of your considerations as well?

Yes. I think the reporting—. I'd say the kind of detail—. It is at that stage we were talking about a lot of—. A lot of the discussion was around anecdotal evidence and what Estyn and Edinburgh university and the children's commissioner—. Now, in subsequent reports—I've got to be careful here. The subsequent reports, actually, highlighted some of these as important issues, so I've got to be careful not to step too far into what was published after my time, if you see what I mean. But, certainly, there was general agreement about the themes that eventually appeared in the framework for action.

Okay. That's helpful. Perhaps I could follow that up with you, Professor Pugh, in terms of what the advisory group did or didn't do in terms of picking that particular issue up.

Yes, of course. The first thing was that we commissioned a report by Siarad Da that went out and actually looked in local authorities and discussed with local authorities and with schools and PRUs the processes that they used, and what they would find as useful and helpful. And it wouldn't surprise you that it came up with recommendations that the actual referral process should be standardised, and the registration process should be standardised, and a number of options for that. So, based on that then, we are looking also at the work that we're doing at the moment about commissioning, and we are hoping to put this together—not hoping, we will put it together—in some guidance that will be, hopefully, going out for consultation between this spring term and the summer term [correction: our expectation is to consult sometime this year].

The variation was enormous, as you mention, in the data. I think one of the clearest pieces of evidence for this was a report carried out by Estyn that was published in October 2019. It's one of the thematic reports into pupil registration practices. And what it does actually recommend is a number of practices for schools and school governing bodies to question at their level where these pupils are going to, because it does talk about a lot of evidence of pupils repeating year 10. And that has grown, I think, in the last six years, three times, the growth of children repeating year 10, and in many cases not therefore going into year 11.

I think that's significant. Also, the movement of young people in year 11 out into dual registration where the main registration is a PRU or EOTAS. All these trends—. So, they've got to be looked at first of all at the individual school governing body level. That's a key point. But the local authority has to cover the data as well from their side, and then, nationally.

And I think the key thing we mustn't miss here is the only way that all of this will happen is if accountability brings in inclusion measures. We've got to have an element of national accountability where schools, local authorities, the Government are held to account for inclusion as well as attainment, because not losing sight of the fact that these young vulnerable people need to attain, but that their pathway to it might be a lot longer than the normative pattern.

Of course. Okay. Thank you for that. Again, Ann Keane, can I ask you whether you found that local authorities were commissioning from private, potentially unregistered or illegal provision?

We received a 2016 report from Estyn that actually says that some local authorities and schools were commissioning EOTAS services from private providers, unaware that they were operating unregistered, potentially illegal provision.

11:10

So, how could that happen? Was it just there were no checks done on them? Or, they were just—.

Well, this is this issue really, because there was no approval or accreditation process and neither was there a quality assurance follow-up to monitor what was going on. I have to say that, for a period of years when I was chief inspector, it was not uncommon to discover, say, in an inspection of a training provider, that there were key stage 4 pupils there and they were in alternative provision. And there is an issue here about alternative provision from schools, which is different from alternative provision from local authorities, in EOTAS.

So, a school can commission and pay for so-called alternative provision even though the legality of that is not always clear. But they could do it under 'learning pathways' at one stage—you know, the 14-19 learning pathways. But we did occasionally visit a provider where there would be local authority EOTAS children and there were people in alternative education, referred from schools, and neither was getting a satisfactory experience.  

So, we had quite a bit of anecdotal and Estyn report evidence of this use of part-time independent schools, of course, which was the discovery that Estyn made, because the registration process for independent schools offering full-time education is quite stringent and Estyn engages in and supports Welsh Government in that registration process and informs it. But unless they are offering full-time education, they don't have to go through that process. So, Estyn revealed potentially quite a problem here for us in Wales. 

So, is that still potentially a problem? Or has this been resolved now? 

I know that the Government, because Welsh Government are—. So, they proposed a requirement for local authorities, didn't they, to establish commissioning frameworks for EOTAS that would have a set of guidelines for providers to have approved status? Has that happened yet? That hasn't happened yet, as far as we know?

It's not fully—. The main issue—. This ties very much up into the guidance that we're going to be putting out for—. It's commissioning and referrals. The two are very much the two sides of virtually the same coin. And that's why I think it really is important, because there's a huge danger in having an unregistered or illegal, shall we say, provider. 

So, can I ask you about the advantages and disadvantages of providers being required to have approved status? So, you might both have a view on that, but I think my particular question would be how the advisory group might be taking that forward.

The advantages of the approved status—the plan to offer approved status—was really that it gave you assurance on due diligence for safeguarding and for quality of provision. It offered, as an outcome, a list that would then be able to be used by referral panels as to the nature of provision, and an assurance that that provision was dependable, and obviously there would be requirements for ongoing monitoring to keep the standards high. So, there are considerable advantages in this system of approved status for those pupils who find themselves being referred to these settings.

The disadvantage, I suppose, would have to do with the costs involved for some settings—not all obviously, but for some settings—in ensuring that accommodation and facilities were good enough, were safe, that they were purposeful, and ensuring that there were qualified staff to offer the pupils a worthwhile experience. 

From our angle, I would endorse everything Ann has said there. It's this process and we shouldn't underestimate the safeguarding, the quality assurance of the data collection. Looking at where this is in this young person's education pathway, because this is a service being provided, it isn't the main provider, or shouldn't be the main provider until that child finishes statutory education, it's looking, then—. That's why, for me, this commissioning, referral, and registration guidance is going to be so important, because it's going to be the one piece of documentation that will cement this together. And, if necessary, as we move forward into the longer term, it may be that that has to become much harder in terms of its implementation, and lead to legislation through that process, but initially that's got to be the process that we move for because it just doesn't exist.

The one thing I would emphasise again within that, that all of that needs to be monitored. It needs to be part of the day-to-day monitoring, right through the three levels of the system—from schools to local authority to Government—because if we don't think that this is valuable, then it's not going to happen and we'll go back to where we were.

11:15

Yes. It's a small group of children statistically, but a group of children who are very vulnerable and in a first world country.

It's really—. It's wrong.

And a lot of them are children with additional learning needs—the majority.

In fact, additional learning needs and those entitled to free school meals, we know, in terms of Estyn's information, are the ones who are the most subject to be repeating years, not completing education. And, of course, we don't know about the other groups that data isn't there for, like young carers.

Yes. Well, I was just going to ask about data, actually, and the difficulty that you've found. Certainly, the task and finish group found, in collecting data from PRUs—I mean, how robust was it or not?

Well, generally speaking, I can probably say that it was not considered robust. There wasn't enough data. There was some evidence of data manipulation. I mean, the whole business of registration to the advantage of schools reporting on performance, as has already been referred to by Brett. I think I would say that in terms of all those processes we've spoken about, really, it needs a process of professionalisation and standardisation across Wales. We're not a big country, we should be able to do this. It's a small number.

There is also the issue of the children who may be off-rolled and maybe don't end up in EOTAS. What happens to them? I think it would be worth looking at the data that we do have and follow a cohort through the system—say from year 7 to year 11—look at the data from schools by local authority, say, look at the data in EOTAS for that period—the same cohort—look at the YOTs data, and then see whether they match up. That's a bit of data mining, I think, that could be done with benefits.

Does that help us to establish whether young people are actually lost through the system?

It certainly is something we should be doing, but I think we should be doing it as a matter of—not as a one-off; this should be a process that local authorities collect data on. They should be monitoring this data, and that's what I really want to see coming through our commissioning process, not just a one-off study, because the danger with anything like that is if it throws up information, it can be swallowed.

It's got to be constant monitoring. The thing is, everyone will say that this is a small group of children and it takes a great deal of time. With young people, children, it takes a great deal of time to do it. But it's so important; it's so important for their well-being, it's so important, as well, because these young people, the costs to the country if these young people are losing their abilities and losing—. Because many of them have abilities, it's just that they've had adverse experiences through their upbringing and their childhood that really make it very hard for them.

I think that's the other thing I would say about data is, we need to look at the fact that it may take these children and young people a little bit longer to get their GCSEs, to get the kind of qualifications they can. Not all of them will be able to get those qualifications, but that gives them that independence and it takes them away from the potential of a life in crime and all the other things we know about, the county lines, and all the other processes.

Anecdotally, on the task and finish group, we had several examples given to us of pupils who'd end up in PRUs or in EOTAS with very low attainment levels, who had issues with managing emotions and behaviour, and the complaint was that there was no accountability measure that reflected the progress they made in that setting, what kind of progress they made in well-being and in gaining some academic qualifications too. But you can't measure them by the GCSE performance indicator per se. You really have to look, with these children, at how well that setting is doing is reflected in the progress they made from when they entered and whether they reintegrate as well.

11:20

And that's a key, I think, as we move forward into the new curriculum. It's centred around four purposes, as opposed to individual subject content. The whole start is this inclusive, holistic approach; it gives us that opportunity.

But the only thing I would stress is it's really important that they get to a level where they can get the qualifications they're capable of doing. So, we must look at that value added, we must understand it, but it must never replace them getting something at the end, and I think that's really important. In fact, that becomes the value added, I think.

But it's also about their emotional and mental well-being as well, which is hugely important in those settings.

And, again, it links—. Sorry. The adverse childhood experiences work that's been carried out, and I think it's going to be rolled out by the end of this term to all schools—the initial development work on it—is really, really important. I can't stress—talking from many different roles that both of us have had in the past with young people who are in this category, it's really important. That's the point of the join up, I suppose, and why we've become more advisory, because we've got to make sure—. Sorry, I bang on like a broken record about it.

Okay, thank you. Did the task and finish group or the delivery group have any concerns about learners' access to a broad and balanced curriculum?

Yes, we did. We had discussions about the curriculum. Now, in 2015, 'Successful Futures' had just been published, and we felt strongly that there shouldn't be blanket disapplication in future of the curriculum. That was one thing that came out strongly. I think it found its way into the framework for action in the end.

But we'd also made the decision that, apart from recommending that EOTAS be included in pioneer schools—and Tai, the Tai PRU was included as a pioneer school—in what was going on with the curriculum reform, we didn't feel that we could actually start a discussion about the curriculum, because the whole of Wales was starting a discussion about the curriculum. So, we didn't discuss the curriculum—there was no point in discussing the national curriculum, because we knew it was going to be phased out, but we didn't know enough about how 'Successful Futures' was going to be developed. But we supported the principle of subsidiarity, and we certainly supported the inclusion among the six areas of learning and experience of health and well-being. We certainly thought health and well-being would be something that, in EOTAS, would be delivered at some level.

Okay, thank you. And did the task and finish group or delivery group have a view on how the new curriculum would be able to benefit EOTAS learners?

Yes. It was very much that the starting point has to be understanding the four purposes of the curriculum and, as Ann said, this well-being aspect AoLE. For us, the important thing there is to make sure the sector understands this huge change, because it is a momentous change for everybody in the sector, in teaching in Wales—that they understand it, that they understand that you can't, if you like, disapply people in the way that would have happened before; you have to look at a very different pathway. There have to be much closer links now between PRUs and EOTAS and schools.

And, again, I would like to see this as part of the staff development model, that PRUs are now included in the funding, the additional funding, for this work. It's something we're really pushing for at the moment. I think there's a study being undertaken by the curriculum division in Welsh Government—I don't know the details of it—into the costings of this for the EOTAS area, but it is going to be really—. We've not missed an opportunity to raise the consciousness of it through our annual conferences.

And the key thing, the other side of that, then, is, if you like, the accountability measures. They must—

11:25

Just before you move on, can I just ask about the whole-school approach to mental health? You know that that's being taken forward by the Welsh Government, and some of us have been pushing hard for EOTAS to be included in the guidance. How important do you think it is that that whole-school approach to mental health is just as applicable to EOTAS provision?

Gosh, it's absolutely key, that's the only way I could say it. I think, in the paper I've submitted, I said right at the—. Looking from where we are at the present time and into the future, this has got to be a key thing. For example, the ACE, adverse childhood experiences, information, has got to be central to understand why these young people are likely to behave in the way that they do, not perhaps to embrace learning in the way that—. And, from the mainstream schools' perspective, as a starter, we want to try to keep them there. There should be, really, very few people at PRUs, if at all possible. It's not a mainscale—it's not like a school, it's not another version of a school, it is a service to a school. And, for me, that's so important—to understand what that's about, or you will never get that young person towards getting the qualifications that will make a change to his or her or their lives.

On these issues of these initiatives, it's important here I think just to mention that there are a lot of initiatives that will be affecting this sector as well as the schools education sector, and the issue there is who is joining up these initiatives where— who's joining them up strategically? Because they go with funding streams as well, and to get away from the silo mentality here is really important. So, I think that's a question worth asking of these initiatives: how do they join up? Collaboration is the key. In terms of the well-being of future generations, I cannot emphasise enough how important it is to EOTAS to be working with schools to open up opportunities, offer greater flexibility, and for initiatives to be joined up.

I would absolutely endorse that. That's one of the reasons why the group has become an advisory group, because it must look at this wider perspective. Otherwise, what we're going to get is a narrow focus on, perhaps, a commissioning framework, but nothing beyond that framework. It's so important.

To what extent did the task and finish group or delivery group consider how EOTAS providers would be able to deliver the new curriculum?

I think the challenge, often, for EOTAS providers is that they're at the end of this continuum of provision for pupils who need a lot of support, need intervention on, as you were saying earlier, management of emotions, behaviour, managing their own lives. I think that the—. I would say the four purposes are central; they are central to the whole, the cross-curriculum aspects. But also, I would say, of all the areas of learning and experience, I think health and well-being—. Now, the curriculum offers a lot of flexibility, a lot of what is called subsidiarity—freedom to shape a curriculum that meets the needs of particular children at particular ages and according to needs. I think this curriculum offers enough flexibility to be extremely useful. I think PRUs, for instance, will not have all the facilities to deliver, say—all the facilities needed to meet the needs across all the areas of learning and experience. That's where collaboration with schools comes in.

I used to find when I was a general inspector of various schools in north and south Wales that there were some secondary schools with on-site pupil referral units. There's no doubt that, geographically, this made it a lot easier for some of those pupils in the pupil referral units to cross the yard and join in some of the GCSE groups. So, there are practical issues like that in terms of enhancing the choices of those children. You cannot expect a PRU with a very small number of staff to be able to offer the whole range of the curriculum. But, as I said, flexibility, subsidiarity, focus on the cross-curricular, the four purposes, and on health and well-being as a good start—and then look for partnerships to offer further opportunities.

I think that's a key—that's a key in the preparation of the staff of PRUs. They should be working with the cluster groups of schools in the training, the additional INSET day, et cetera—that whole process, working on that. And I think again it's about also thinking for the future of PRUs through the twenty-first century schools funding, which we have got representation on now, so that bids do go through that are creative around that.

I would agree with Ann, when you have—. In a previous life, I was a director of education in one authority, and we had eight secondary schools and they had an internal—it wasn't called a PRU, it was just a referral centre, but they were kept in school, if you see what I mean. It was a temporary movement and then back in. It was expensive. That's the issue. It's got to be looked at and staffed with a proper structure, rather than just anybody who happens to be free for that process. 

11:30

Thank you. My next question relates to what you said, Ann, about the good practice you'd seen in some places where there was joint working between schools and PRUs. Is that very widespread? Can I ask both of you: how widespread is that? Or is that very much an isolated example that you gave?

The examples I saw when I was actually out visiting schools a lot, the best ones were where the schools were geographically sited—the PRUs were with the school, or part of the school or near the school. Simply because of the practical issues of getting to classes in other subjects. And so I would say that there's not—. Certainly, at the time when I was chairing the task and finish group, there was not enough collaboration with different providers, there was not enough staff development, there were not enough staff in PRUs to easily enable the release of staff to go to professional development events. That was the situation at that point. And, of course, some PRUs have a very unhappy history. I have to say, though, that, even in 2015, when we were writing that section on the weaknesses of PRUs, we had certain PRUs that were doing really well and still are. And when that happens, that always raises my spirits, because you can begin to think, 'Oh well, it can't be done, then.' Well, it can be done and it is done, and Estyn's reports still show us how it can be done. So, that makes me feel all the more optimistic that it's possible. 

And I think the best vehicle for trying to ensure that this is done is the new curriculum and to look at the accountability measures that go with that, because I think there will need to be a qualitative form of accountability, not just a percentage, which is always the easy thing to see but hides a multitude of sins. There needs to be a quantifiable form that needs to be monitored by the consortia, the local authority. I think that's the best way of making mainstream schools work with PRUs, when those schools are going to be held accountable in the same way as they are for GCSE results, shall we say, now, or end of key stage or reading test information, when they're held accountable for how inclusive they've been. And I think—. But that will be a hard measure, because it will have to be quantifiable. But I think, certainly, things like introducing destination data—I know Ann's mentioned that—where do these young people go to, is going to be really important, and that is quantifiable, obviously. 

And when we look at—. Certainly, when Estyn inspects special schools, they look at distance travelled, they look at the support and how well those pupils are faring in terms of the support they get. I think there's a parallel there with EOTAS. It's about the distance travelled.

And there's some tremendous practice, isn't there, in special schools— 

—in terms of how they look at small steps, if you like, but which are huge in terms of that person's personal development. 

Yes. They're qualitative, and they wouldn't compare with mainstream GCSE outcomes. 

Okay. Thank you. 

And, for EOTAS providers who are not PRUs, have they got access to services from regional consortia? Do you find that there's any resistance, either from the provider or the consortia, to having that relationship?

Going back three years, what we were unhappy about in the task and finish group was that there were not enough challenge advisers with the depth of experience that would enable them to operate in EOTAS, and we wanted the consortia to do more to offer more targeted, focused support to the sector. 

I think there have been some moves forward in terms of, for example, pupil development grant funding. They're now within the consortia for officers. One of them has a speciality for EOTAS provision. It's early days. I think we would need to look at how effective that is really going to be. I think the information—. It's the information flow that's going to be important, certainly to the private EOTAS providers—that they know that this training, this development is available. But, of course, so much development these days is carried out within school, and that's the key thing therefore, bringing it back to the fact that there needs to be working across providers with mainstream schools, because that's where the bulk of curriculum development is precisely because it is the new curriculum, and it has been developed the way it has.

I think the key point is co-ordinating on the part of the consortium, but getting those providers into partnerships with schools. That will provide problems because they are commercial suppliers, if you like, and in competition with each other, but that will be for them to—. I think, again, it will come on our accountability measures. If they are found wanting—. It's a very negative way of putting it, but I'm sure that that will be the biggest way of sending them into that work. 

11:35

Can I ask on that—? In EOTAS, there's a disproportionate provision that comes through the private independent sector. The Minister has already told us that Hwb resources will not be made available to the independent sector in a different context. Are you a bit concerned about that, particularly as that's where the majority of the new curriculum resources are going to be made available to teachers? 

My issue with that would be for these people to have access to the information, whether it's through Hwb or through a different area. 

Yes, because otherwise, they will end up in a situation of saying, 'I'm not coming up to the mark'. 

Whether they go to their own regional consortia lead and ask for the information, you know: 'What is going on in this part of the world? What can we actually therefore engage with?', which might be a more direct way even than going via Hwb for each area. Because, of course, there are issues around things like Welsh language as well that we've talked about. Each consortium is going to have different strengths in that way. So, I would say what they need to do is go to the lead person in their consortia. 

Can I just ask about primary legislation, then? Have you got any views on the desirability of the Welsh Government developing primary legislation to include the introduction of new organisational structures for PRUs, and what would be the advantages and disadvantages of such an approach? 

There was a strong feeling among, I would say, probably a minority of the task and finish group, that PRUs should be more like schools; that they should have the autonomy of schools and that they should therefore have their own budget because there are financial constraints in the context of a local authority spend. But I have to say that, personally, I feel that if you take Estyn's model of the continuum of provision, and if you take the point that Brett has made that this is definitely a service, not a school, I take the view that keeping them as they are, with certain guarantees that they have access to training, support and challenge, and that they are professionalised, if you like, that that is a better way forward than making them into schools. 

There was a strong voice in particular that wanted that autonomy for teachers in charge to be headteachers, and for management committees to become governors, and so on. But, on balance, I think for the reasons that we've already explored, they are better off as they are—a service provided by the local authority. 

I would endorse that. I think it's very important for us to see them as they are. I think it's also important for us to monitor closely how they deal with the reforms, the whole panoply of reforms that are going on at the moment, because that's going to be important for us, but not to leap. If we leap in now—. And certainly, if we put up, for example, PRUs or other EOTAS providers as becoming a school, then that defeats the purpose because the whole purpose is to keep young people in mainstream schooling, rather than in this—.

I think what is important, though, is that we do have a career structure for staff who work within PRUs and EOTAS and that it is seen as a job that counts, that matters, rather than—. 

Thank you. And what about the pace of implementation of the framework for action? Are you satisfied with it? Does it need to go much quicker?

11:40

Brett has made the point about the need to bring in the developments, in terms of EOTAS, in parallel with the curriculum reform. And I think it's a very good point. I would say, though, that it's almost 10 years now since the Edinburgh University evaluation was commissioned, and we really haven't seen many of the changes that we've been discussing here this morning. 

Certainly, I'd add to that that, as I said earlier, I would want this to be going faster, but we have to be very careful as well, within the current state of change that we actually have, that we're looking very carefully; that we are not setting up something that we might regret in two years' time. We've got to really consider it in that sense. But, I would agree generally that this needs to be seen; it needs to be on people's agenda. It's a small group of young people, it's easy for them to slip off that agenda in the bigger national picture of things. 

And that's why we're doing this inquiry—to make sure that they are firmly on the agenda. 

That's the thing I would want to stress over and over, because it's so easy for them not to be counted in training funding, not to be counted in any kind of new pilot that's coming out. 

And it's not just the ones who end up in EOTAS; it's the interventions required for schools for the ones who might end up in EOTAS. 

Precisely. That's the real thing. 

Thank you, Chair. It's really following on from that, and whether either the task and finish or the delivery group has given some, or gave consideration to the amount of financial support that learners in EOTAS actually need. So, you touched briefly on the amount of support needed for the new curriculum; you talked about the fact that this is a service, and not a school, so that there are all sorts of constraints around this, but did either of the groups give some consideration to what you think is actually needed in financial terms for these learners?

No, we haven't worked on the actual costings of this process, because I think it would be quite difficult; the whole curriculum roll-out has been developing during that time. What we think is important is that the structure should be there to provide that funding, and we would see that as coming through a commissioning framework. When local authorities have a commissioning framework that is standard, or near standard, then that should mean that the money should be following around the young person, whether it's held by the school or by the local authority. But that's—

Yes. We need to have that structure there, and then, whatever the costs would be for somebody in mainstream, we'd be able to extrapolate the information from that. But, without that structure, there's no guarantee if we said 'It will cost X', that it will actually happen. 

You need the data on the costs and the actual costings linked to the range of provision, to be able to project your budget needs. 

And the costs are quite variable as well, I would imagine, and can be variable from year to year, term to term, I guess, in that sense. Okay. 

My final question, Chair, is just whether the Welsh Government's accountability and performance measures perversely incentivise schools to off-roll pupils. I think, Professor Pugh, you kind of touched on that very briefly earlier on, so, perhaps you can just expand on that a little bit in terms of—

I think, if you want to see perhaps some of the evidence for that, this report I mentioned, the Estyn report, in October 2019, if you look at the actual main findings before you go into the main report, it actually says most explicitly that there is a great danger that what schools will do with this high-stake accountability is look at data alone rather than first-hand evidence—the classroom observations, the learning walks, the other—. Both have to be there. 

And, to a certain extent, perhaps this system in Wales needed that accountability drive at the time that the push for data came— at that time—but a more sophisticated system, as you go further—. If you look at the Organisation for Economic Co-operation and Development findings, very often, you start at the point of ratcheting something into improvement. You get some unintended consequences from it, but that's where the most sophisticated system—and I would say we are there now—needs to come in. We need to be looking at that qualitative data as well as quantifiable. But, unless it's there, unless schools are held to account for that data, unless local authorities, the consortia and Welsh Government themselves are, then it will go off the horizon. And it's always much easier to look at statistical data when people are doing—.

11:45

Sure. So what would be the—. Obviously, the data has to inform everything.

So, what would be the specific measures that you would want to see in place?

These would have to be much more processes about school staff evaluation, so you'd come in at that level, that would need to be monitored by the consortia challenge advisers. And I would like to think, in terms of the new framework, that Estyn will be involved in that, because, obviously, Estyn are looking at a new framework for inspection, and that's why it's so important. It's the kind of thing you can't just look at in a table, for example, sat in Cathays Park—you couldn't look down at tables; you'd have to be in the school. It's a bit like the equivalent of the medical gaze—the thing that the doctor does, looking at you, rather than just the data that flow from it. Some of it will transfer across into data; we mentioned destination data—that's one bit. But a lot of it will be about how pupils are included, behaviour and modification processes, and how schools use projects such as ACE, which is one we've talked about; there are many other similar projects—how they're using it, how they're implementing it, how they're measuring it. If that helps.

The metrics on quantity and quality, they are different. I don't think we're ever going to get away from metrics on quantity, to some extent, but the metrics on quality need to be there. And that's about distance travelled, valued added, it's about looking at other aspects of pupil experiences, and quantifying that, and using that as a measure. So, we've had too much quantity metric accountability in Wales—the categorisation, even the grading of inspections, the performance indicators—and they have driven unintended consequences. And the off-rolling, and the year 10 repetition, has been to do with that. So you've got to step back and rethink, yes, some quantity metrics, but quality metrics—how do you measure, how do you capture that quality, especially in EOTAS? Because it's the quality of that improvement in the pupils you want to capture.

Okay, thank you. We've come to the end of our time. It's been a fascinating session, so thank you very much, both of you, for attending. I'm sure everyone has found it as useful as I have. We will send you a transcript to check for accuracy following the meeting. Thank you, both, again for your attendance. 

4. Papurau i'w nodi
4. Papers to note

Okay. Item 4, then, is papers to note. Paper to note 1 is a letter from the Chair—that's me—to the Economy, Infrastructure and Skills Committee, updating them on our budget scrutiny session on 8 January. Paper to note 2 is a letter from the Minister for Housing and Local Government regarding clarification about indicator-based assessments, and their purpose within the local government settlement. It offers us the opportunity to have further discussion and clarification on that, which we can return to shortly, if that's okay.

5. Cynnig o dan Reol Sefydlog 17.42(ix) i benderfynu gwahardd y cyhoedd o weddill y cyfarfod ac ar gyfer y cyfarfod cyfan ar 30 Ionawr.
5. Motion under Standing Order 17.42(ix) to resolve to exclude the public from the remainder of the meeting and for the whole meeting on 30 January

Cynnig:

bod y pwyllgor yn penderfynu gwahardd y cyhoedd o weddill y cyfarfod yn unol â Rheol Sefydlog 17.42(ix).

Motion:

that the committee resolves to exclude the public from the remainder of the meeting in accordance with Standing Order 17.42(ix).

Cynigiwyd y cynnig.

Motion moved.

Item 5, then, can I propose, in accordance with Standing Order 17.42, that the committee resolves to meet in private for the remainder of the meeting, and the whole of the meeting on 30 January? Are Members content? Thank you.

Derbyniwyd y cynnig.

Daeth rhan gyhoeddus y cyfarfod i ben am 11:48.

Motion agreed.

The public part of the meeting ended at 11:48.