Clinical Research Training: A Blended Approach

About the Advarra In Conversations With…

The future of healthcare innovation hinges on research and clinical trials. Advarra sits down with leading experts to dig into pressing issues and explore cornerstone solutions. Join us as we discuss topics and trends impacting the healthcare of tomorrow and advancing clinical research to be safer, smarter, faster.

 

Transcript

Chapter 1: Onsemble Recap

00:00:03.260 –> 00:00:21.270

Hi, everyone, and welcome back to the Advarra In Conversations With podcast, my name is Eric Sperling. I am the Technology Training Manager here at Advarra. I’ve been here over four and a half years now. And I’m thrilled to explore our approach to training with Jonathan Aguilar, the clinical research training specialists at MD Anderson Cancer Center.

00:00:21.270 –> 00:00:45.240

Hello, Hi, Eric. Thank you very much for having me. I’m very excited to be here today. As you mentioned, I work for MD Anderson Cancer Center as a clinical research training specialist. I’ve been with them now for over a decade. And I’m excited to share the knowledge that we’ve gained after working on this OnCore implementation and how it’s gone at MD Anderson. So thank you.

00:00:45.240 –> 00:01:17.130

Awesome. So let’s let’s jump in. I know we have a couple of topics that we’ve kind of want to touch on. And then we’ll see if we have time to get to a few more things. But first and foremost, this last March, we were able to actually meet in person and you know, you and I have been working together for it seems like almost a couple of years now working on the MD Anderson training project and getting that whole OnCore implementation going. But first and foremost, at Onsemble, I just wanted to ask you like, what were some of the big takeaways you had attending, in going in person live meeting other people at our March Onsemble?

00:01:17.130 –> 00:02:43.050

Well, it was absolutely an amazing experience. And I’m glad you brought up Onsemble because it was such an impactful experience to go in person. I hadn’t been to a conference in such a long time because of the pandemic. So, it was nice to get to see everyone, a lot of people that I had been working with previously and meet all of these new individuals, professionals from all over the country from all these major big-name universities and research institutions that are working with this platform. And helping us understand what they did, how they did it, what were the differences, obstacles they encountered. And then actually attending the sessions and learning about the specifics of their programs or the specifics of their universities. It just blew my mind at how amazing this conference was. I came back, absolutely thrilled about it. I’ve started to implement some of the things that I learned. It was definitely an eye-opening experience, and I would again love to continue to go to these conferences, I came back just essentially thrilled and making sure that that my superiors knew how important this conference was, and that we needed to send more people. Hopefully, I’m not overselling it. But I just really had a blast at the Onsemble conference. And I highly recommend that anyone that’s working with Advarra, OnCore, anything related to go to these conferences, because it’s such a huge help.

00:02:43.050 –> 00:03:29.090

Absolutely. And I know I’ve gone to a couple of myself. And it’s so impactful in the sense that not only is there so many sessions on training, I know that’s what we primarily focus on and that’s what we’re going to be talking about for the rest of this podcast. But there’s so much more beyond just OnCore and it’s just really talking about and getting down to the one-on-one level, as you said, introducing yourself to different people finding out what their struggles with their successes are and then meeting the keynote speakers and speaking to all these different levels of professionals across the entire industry and really making that one on one connection. So, speaking about the training around that we live and breathe in every day. What were some of the major Onsemble talks or discussions that you attended that were impactful? And that really made you thrilled? Like you said.

00:03:29.090 –> 00:04:06.160

There were definitely a lot of presentations, a lot of sessions that was absolutely amazing. The two that I really felt made an impact or that I remembered a lot. One was the simulation-based proficiency testing for end user training. And that had to do with the fact that the gentleman that presented Mr. Marlon Abrazado from UCLA hit on something that I work on all the time. And it’s so important that the online modules that we’re creating have to have a better, more developed system to help understand the proficiency of the users.

00:04:06.160 –> 00:04:09.000

More of an interactive type thing?

00:04:09.000 –> 00:04:47.170

Exactly. But to build something interactive to understand how the individuals are understanding, are they actually understanding? He basically put it very simple in one of his slides. He’s like, are we training for a test? Or are we training to measure for proficiency? And that just, it put it in a different perspective? For me, it’s definitely something that we should all be looking for the proficiency of the end user. Are they grasping? Are they understanding? Ultimately, that’s the goal of an instructor of an educator, of a trainer to make sure that your user is fully understanding what they need to do, and why they’re doing it.

00:04:47.170 –> 00:06:05.290

And I think that’s a great point that you bring up with that because, especially as a trainer, we only have so much power to make sure that they actually do the thing, a lot of times, it’s going to go to the department heads, and then it’s going to go to leadership do to really help enforce or engage the workforce in that adoption of new technology. And I know, as we progressively get older, sometimes adopting new software and new technologies isn’t always the most fun, but it does have major benefits. And I think that that talk by Marlon was fantastic in the same sense for me is, how can we make a safe space for people to practice and for them to be able to engage with the software, make mistakes, get quick, instant feedback so that they can mentally make those changes? And it’s not, you know, a couple of days down the line. So like you said, we’re not teaching to a test just to say yes, according to my training methodology that they’ve been okayed, and they can move on, it’s really more of, can they actually do the thing, because when we’re talking about clinical research, we need to make sure that the information is entered in properly and correctly, because we need that information and that output later on. And so it’s not just Yep, they did the thing, or they took a couple question tests. I think they understand it, but I haven’t actually tested the proficiency. But I think that’s absolutely a huge point that you bring up and that was a fabulous talk about Marlon.

00:06:05.290 –> 00:07:19.180

Yes, yes. And the other aspect that I loved, and this has to do more with the in-person component of actually getting to meet these professionals, is it also allows me the opportunity to question well, how what’s your workforce look like? How many people were involved? How long did it take you to do this? It sounds like an absolutely amazing program. But if I don’t have the resources to do it, or I don’t have the knowledge base to do some of the things that you’re trying to do, where can I get it? How can I get it done. So, there was so much that I felt I gained from his presentation. And he was a connection I was able to keep we connected on LinkedIn, and I communicate with him as well. So, it’s something that I highly, again, highly recommend to everyone to attend it. But that was just because he does a lot of the same work that we do at MD Anderson as far as creating educational content, not just necessarily for the OnCore platform, but really just in general for the needs of our institutions. So that was definitely one that really stuck in my mind. And then the other one that I took away from was the adaptation through adoption presentation by the University of Utah by Miss Edvalson and Miss Turner.

00:07:19.180 –> 00:07:22.130

Is that the one she started off with the paper airplane, right?

00:07:22.130 –> 00:07:24.020

That’s exactly right.

00:07:24.020 –> 00:07:24.270

Yes, that was fun.

00:07:24.270 –> 00:09:57.160

That was a very interesting approach to presenting. It definitely engaged all the audience in one get go. And I think we even had a huge group picture. I haven’t seen that group picture. I hope they share it. But it was absolutely amazing to start that presentation that way. They took their time in talking about their experience implementing OnCore what it was like for them. Something that caught my eye or that I had heard a lot of while we were working on OnCore, the implementation strategy is that we were a large institution, and they were training a lot of people. But I had no frame of reference of what’s considered large or what’s considered small. And they talked about that in their presentation. And it wasn’t till I went to this conference that I finally understood a little bit about what people were saying about our numbers being so large, now we have a large training team as well. So, we had the support in place to make sure it got done. But how other institutions are managing with less people is amazing to me, because there’s so much work, there’s so much planning that goes into it. And I believe if I’m not mistaken, if I understood correctly, Utah had four people for the entire implementation as far as trainers go, we had a much larger training team at MD Anderson. Granted, they did train roughly about anywhere from 10 to 20%, of the volume of individuals that we have to train at MD Anderson. So, their population is of employees that needed the training was much smaller by comparison. But still the level of work needed to implement this, it’s, it’s a big task, it’s a big endeavor, highly important, highly needed. I definitely see the potential that this platform has to advance our work. But it is something that it takes time, you have to go through planning. And just like you mentioned, at the beginning, it seems like it’s been a while since we’ve known each other, it’s looking back, when we first started talking about implementing this in the initial meetings, it’s probably been about two years. Of those two years, the first year was strictly planning. meetings, understanding what it was going to take, how much it was going to cost, things like that. And then the next year after that, the second year, it’s been a lot about training and the different phases of our implementation strategy, which we can get into later. But again, absolutely amazing experience at Advarra highly recommended to absolutely everyone that is working on this platform. And it’s an invaluable experience that hopefully, I get to repeat in 2024.

00:09:57.160 –> 00:12:46.050

Absolutely, I’m hoping to see you next year as well, in Arizona. And just to wrap up that quick topic too, because one of the biggest things that I remember from their mockup was they had a picture of a multi person bike. And if you have a multi person bikes, a five seats on it, not everyone is always going to be pedaling, if you can get a couple of like we like to call them champions or super users in the technology to help champion with a cause and how it’s helping them and to be able to relay the benefits, that’s always going to be huge. And the big thing is to make sure that people aren’t putting on the brakes, if they don’t have the feet on the pedal that’s not always the best, but they’re at least not slowing down the process. And going back to kind of what you were touching on with your implementation, we can transition to that topic because I made this joke at Onsemble but you had to train a village is really what you had to do. So if we can kind of go through touch on just briefly kind of what phase one, and I’m just going to throw at some of the numbers for everybody that’s listening to this, of the scope. And when we’re talking about the scale of what MD Anderson Cancer Center had to go through. Just starting with figuring out who needs training. So it sounds like great, we have this new software technology, we need to implement it. What do those meetings look like? And that’s what you’re talking about is having that whole year of trying to figure out what departments need it, what areas need it. And the thing is, is a lot of it comes down to training at the end of the day to say, yes, we can actually utilize this technology. So, with Phase One I know and I’m just looking at some of our former notes from back when we were able to speak together on this topic. Roughly 23,000 employees had to be sorted through to say, okay, out of these 23,000 employees, how many actually need training? So what did you have to go through to figure out from 23,000 to get to that final number. So what you had mentioned is that you had to do HR cross referencing, you had to take a look at who was in the old system to bring forward and the new system, but also who’s gone out of the old system into the new system, which is a lot of data analytics and sorting through that information. From there, you had 75 departments. So, then you had to figure out in each one of those 75 departments, how many people actually needed to be trained, who were going to be your subject matter experts, who were going to be those that were going to be your champions. And at the end of the day, you roughly had about 1,100 end users that you had to train and if I get this number, correct, was it 25 or 26 business days, so excluding weekends, you and your team had to train 1,100 end users in roughly 25/26 business days. Which isn’t a lot of time to try to get everybody up to speed on their jobs in the new software package. So could you talk a little bit about that process?

Chapter 2: OnCore implementation Project

00:12:46.050 –> 00:16:21.000

OnCore implementation Project
Definitely, definitely. It’s absolutely a testament to how strong the team is at MD Anderson. As far as how everyone got together to work on this project. It’s something that we now we did have a large team, we had the CTMS administrator and his respective team helping us with the technical side, the training team had roughly about six trainers, in addition to leadership managers, helping pushing, making sure that there were opportunities for us to move forward with the concentration on the OnCore implementation project. Because we did have this team was already existing, we didn’t hire people specifically for this implementation. So that meant that we had to essentially take people that were working on other projects to start working on this on this OnCore implementation, it was our main focus for the last year. And as far as the planning goes, we started like, the whole year before we were having workflow meetings with a wide variety of subject matter experts, from finance people, to regulatory people to in the field, clinic people, managers, directors, etc, etc. Really, the list goes on, we had several of those meetings during that first year, just to fully understand the entire process from the initial idea of wanting to create a scientific project, all the way to finishing it and publishing the results and what that meant. So it was a long process of planning. I think we got a lot of the very important figures. But really, what it boiled down is making sure that the key leadership figures are on board to making sure this is a commitment that we were going to take. We definitely didn’t want to put ourselves in a situation. And I had heard other institutions at the conference, put themselves unfortunately, in that situation, where they fell back on the old platform while they were trying to implement the new platform, and they had delays in implementing the new one, because they kept using the old one as a crutch. We fully committed, we said, this is what we’re going to do, we did take a three prong approach to the training. And that was where we discussed a phase one, two, and three. And during phase one, we focused on essentially new studies that were being opened at MD Anderson and gave everyone the opportunity to learn the system at a slower pace, because we were only focusing on new studies that were being submitted and approved in and those take a little bit longer to get traction, if you will. But it also gives the end user time to learn the system. While we’re going through, there’s a lot less studies that are just brand new versus existing studies. And so it gave us an opportunity to train those individuals. And you’re completely correct. We had roughly about 1,100 end users that we had to train in that 25 to 27 business days. And I actually went back and ran the numbers again to see what it looked like since we talked about because it was a blended approach. We did eLearning modules. Back then we had 5931 total eLearning modules that were completed. Since then, until now we’ve had over 11,925 eLearning modules completed. So we’ve essentially doubled the number of modules that have been taken. But just to show the importance of that timeframe from back then, essentially 50% of the trainings that were completed over the entire span, it’s been roughly about 337 days, 50% of it was done within those first 27 days, within that first push.

00:16:21.000 –> 00:16:21.290

So that’s amazing.

00:16:21.290 –> 00:17:00.230

It absolutely goes to show how much work was put in that first push for phase one. But it was it was important that we take the time to develop the training properly, because that’s also the first time it was introduced to the end user. So, we wanted to make sure they understood what was going on. We took extra time to make sure that not only were they being introduced with the online learning modules, but we also had instructor led training sessions where we got to meet directly with the end users. All of them had the opportunity to attend. And they got an opportunity to ask us questions while we were instructing them on how the new platform was going to operate.

Chapter 3: Blended Training Approach

00:17:00.230 –> 00:18:01.150

Blended Training Approach
Let’s touch on that real quick, a little bit more in depth. So I know you had mentioned that you had a three pronged approach. Phase one was new protocols. So new studies, you’re going to start from start fresh, put those inside the system. Phase two, you’re going back and taking your existing ongoing protocols and going to be putting those into the system. And then phase three is going to be the full implement. It was the full implementation of the financials aspects, and putting that stuff in. But and we’ll get to phase three in a little bit and kind of like lessons learned and what you’ve been able to translate and take with you from phase one. TheGood, the Bad and the Ugly. Because we always know that we always need to continue to improve how we train and how we engage with our audiences. But going back to that blended learning approach, can you talk a little bit more about that because I know you’d mentioned the Elearnings being able to allow them to take Elearnings which again, can be interactive, that can be short videos, it can be flashcards, it can be little quizzes and stuff like that, to help kind of that beginning engagement. But what was that process for you? It was an eLearning, first than instructor led trainings, etc. And if you could just kind of explain that for our audience.

00:18:01.150 –> 00:21:59.060

Definitely. So the bulk of the training again, did happen during phase one. So the way we divided it, we work closely with Advarra for the creation of the custom eLearning modules. Those were uploaded after they were created, depending on the topic, they had different specialties, whether it was general navigation, task list, etc. And off study or on study on treatment, etc, there was a wide variety of different topics. We split them up based on the responsibility. So, we had individuals that had the responsibility of managing and enrolling subjects on studies. And then we also have individuals that have more of a regulatory responsibility that will work with uploading documents to make sure that the system is updated with the correspondence from the IRB, whether the protocol was approved, or whether the protocol is going to be withdrawn, etc, and how to notify your field staff on how updates or amendments have been approved. And as a consequence, there might be changes to the system right within the system. those are the two main ones, but we also had to do trainings for individuals that are using the platform, but they have very specialized responsibilities such as the SRC coordinators, and sorry, SRC standing for Scientific Review Committee. The they have a very special need for the system for understanding how the paperwork gets submitted, reviewed, and approved, etc. And then how it was going to work with our Eprotocol platform, which is something that was developed as well by Hyrum. So we can submit and review protocols work with the scientific review committee with the IRB at our institution for purposes of reviewing those protocols. So there was some slightly tailored training for a few select teams. But by and large, the bulk was our subject management and our regulatory management responsibilities. So from there, we created learning plans for each situation. So a subject management, let’s say, in I can’t remember how many exactly, but they had roughly between five to seven online modules, again, with different topics. And at the end of that learning plan, they had to complete an instructor led training, and regulatory management had something very similar. Some individuals at or institution had to take both training plans and would gain access to both responsibilities in the system, or some just made sure that they attended the training they needed. All of them had to complete an instructor led training, in addition to the online modules that they completed. So, it definitely gave them the opportunity to get an intro of foundation through the online merging our online learning first. And then they had the opportunity to attend the instructor led trainings. In order to ask us questions, they already had somewhat of an understanding from the modules, they would get access to the practice environment as well during the instructor led trainings which allowed them to play around in the training a safe environment, of course, we don’t want actual live production data to be messed around with but in in the OnCore training environment, we granted them access. And they got the opportunity to create reports to learn how to enter information, whether it’s the protocol, data or whether it’s amendments, or whether it’s submitting months or continuing reviews or how to enter subjects how to determine if they are on study, if they are eligible, if they’re consented, etc. All of that comes into play in as far as teaching them so it took a little while to get through phase one and all those people but at the same time, we had an absolutely committed team and shout outs to again the entire MD Anderson OnCore implementation team because that it’s so much bigger than it might sound. But it’s something that they absolutely did an amazing job. And I can’t emphasize enough how much work and commitment they put into this project to make sure it was successful. So that was our Phase One strategy.

00:21:59.060 –> 00:22:12.120

With your instructor led trainings, when you said that they were able to get into the training environment, were they following along something that you were kind of demoing for them? Are they just watching you go through it, and then they did it? Or were they like kind of falling right on your heels.

00:22:12.120 –> 00:26:19.100

So essentially, as the instructor, because I’ve actually taught both of the classes, we have our intro to okay, this is again, what the platform is, this is what we’re doing. This is our training strategy. But then we go through an actual activity where they actually get to have a demo protocol assigned to them for the regulatory management side. And we go through a mock scenario of what you need to do, what are the steps you need to do, if you need to upload the initial approval by the IRB? Where do you go to document that? Where do you go to upload your documents in the system, whether that’s the PC console, or if you’re doing the EPRMS submission for the regulatory management side. And that’s from the regulatory side, but on the subject management side, They are given a demo participant and that demo participant, they follow along. So that demo participant by each person that attends is assigned to a protocol that we tell them, we give them a protocol they assigned to and they go through the process of assigning them into the protocol consenting them, deeming them eligible, where you would go, what are the buttons you need to push? What are the dates you need to enter? Where are you going to go to understand which consent documents you’re going to approve. And as we’re going through the activity, explain how it might be different based on the type of protocols they’re managing, because that was the other tricky part is you have to develop one training system that will work across the board for a wide variety of different protocols. Unfortunately, that’s going to be different. And sometimes the terminology can be a little confusing. So, for instance, we had people that were training with us that were managing behavioral studies. And that’s going to be very different from Leukemia studies, or investigational cancer therapeutics, where they do a lot of phase one trials, they that it’s completely different in how they manage their workflows. They’re all equally important. But my point is in the system, it talks a lot about assigning treatment arms. And for the behavioral studies, they don’t have necessarily treatment arms. But that’s where we had to explain to them what is meant here, they all any participant that’s going to be in registered in the system needs to be assigned an arm in the system. And as a consequence, think of it more as an intervention arm and for their scenario. So it gave us an opportunity to talk to them to explain to them to help clarify some of the confusion and some of the terminology and actually explain the resources that Advarra has made available as well. That little question mark that takes them to the glossary in the system. I think that’s absolutely brilliant. I use it all the time. But you know, letting them know that that resource is available, letting them know about the Advarra learning portal and how to access it and Advarra University and the different trainings that Advarra as well, in addition to what we have developed at MD Anderson, because at Advarra, while y’all have done an amazing job, there’s no way you can customize training for every single institution that uses it. So, it would be it would be impossible to do that. But that’s where we kind of come in and build and supplement with the work that you all have done. And we do customize a lot of our work guides, tip sheets and tricks and online learnings specifically again to help supplement. We want to be successful in this implementation and the usage of this platform. So this sometimes it means adding a little bit more information for certain procedures that relate specifically to MD Anderson so it was a definitely a positive experience in the majority of ways I can possibly say that implementing this and a big part of that is working with all of you and the way you all helped us to implement this and I gotta give props to the Hyrum consultant team because they were with us the entire way. as well. And that’s something that they did their due diligence as far as making sure that we as trainers also understood the steps and where they attended a lot of our meetings as well to make sure that if we were stuck, or we didn’t know how to answer certain things, because we were not familiar with the platform, they kind of jumped in and helped us quite a bit as well in that area.

00:26:19.100 –> 00:28:08.040

So taking that thought, because obviously, your blended approach, the process that you all developed in was taking the eLearnings, whether they were custom, or just more of like the general eLearning, assigning those to staff members that need to be trained, once they completed those areas able to attend an ILT and run through an actual practice, run, or multiple practice runs with an instructor where they can ask and get immediate feedback. And after that, they were given access to the production environment and able to start taking care of new protocols. And then phase two is, is existing protocols. And then what we’re going to touch on here is financials in that phase three. Now I know financials, having worked with OnCore for a very long time, is really where the rubber meets the road, it really brings in a much deeper layer of complexity into the system, because you have to make sure that your specification, your calendar, your schedule events, whichever term you’d like to use, is correct. And then you can start associating all these various costs. Now, I know that we’re not in the realm of clinical research to get rich, but we are in the realm of trying to change the world. However, like I always like to say you do things for either love or money, either you love it, or you want to get paid for it. Well, in our case, like we said, we’re trying to change the world because we love what we do. And we love how we can actually make an impact in the world. But you still have to keep the lights on, you have to keep the doors open. And that’s where that financials segment really comes into play with OnCore. And again, it does add that layer of complexity. So thinking about that blended learning approach in that process that you had from phase one, and then even in phase two, can you talk a little bit more about lessons learned and how that’s kind of translated a little bit more into phase three, which I do believe, I think last time we talked was it was it February, March, April, this recently wasn’t it was I feel like I was very recently here.

00:28:08.040 –> 00:28:12.100

You have no idea how recent we just went live May 23.

00:28:12.100 –> 00:28:15.070

Congratulations! That’s awesome.

00:28:15.070 –> 00:29:22.100

So three days ago is essentially from the date of this recording here. But it’s something that it’s very recent. So it’s very hard for me to say, at this point in time how successful the training and implementation has been for phase three. We don’t have an analysis. It’s a little nerve wracking because things have been relatively quiet. So I’m going to knock on wood. And hopefully that means we’re absolutely amazing. And we’ve done a great job. But realistically, we did learn a lot of things from our phase one implementation. One of the things that we got in our in our feedback after phase one is that the users wanted more practice with the system before being expected to do it in the live production. So that was something that we thought, okay, well, how do we do that? How do we give them more practice? hands on practice? The modules were great. It was an initial foundation. But again, It wasn’t tailored specific enough for each department. And that’s where finding ways to give them practice. It was an additional element, additional challenge that we had to deal with for phase three.

00:29:22.100 –> 00:29:38.260

Like we wish we could have that simulation that like Marlon had actually built out for UCLA, if we could apply that across, but I know how much time and effort that takes but that would be a perfect appropriate part for that. Because then they can go in and you can kind of tailor it like that. But what yeah, what did you end up doing?

00:29:38.260 –> 00:35:01.240

Exactly. So I would love to have said that we had time to create something like that. He did admit it does take some time to develop that sort of training system, if you will, as far as creating that simulation base, because it’s very detailed way of creating modules. I absolutely loved it. But it does take time. It takes a lot of knowledge and understanding and even one of the things that they have more of versus us is they have instructional designers that they’ve hired for purposes of creating. We our team doesn’t have an instructional designer right now. That is something that we have a lot of experts in the field on our team from the IRB from auditors from people that have been research nurses or coordinators in the field in. Now, as far as creating educational content, that’s something that we need to work on improving. But we’ve definitely done a lot as far as understanding the systems and understanding the educational platforms necessary. So whether that be using things like Kahoot, or to engage with the students, or in this case, using Articulate to create a lot of the modules, I know Advarra is using Articulate as well, a lot of the modules that were created, were using Rise 360. We’ve started using the that Rise 360 as a consequence, and we’ve been using a lot of studio and storyline as part of that creation. But getting back to what you were asking about phase three. In that strategy, one of the things that we did is, instead of focusing on the instructor led training side, we focus on essentially after the online module that the individual takes for the subject visit tracking is what we called the learning plan, if you will, in the in our LMS. In our learning management system, the subject was a tracking module, they would complete that, but there was essentially a homework assignment that they had to complete i Excuse me, I call it a homework assignment, because it was relatively long, they had to take some time to go through the instructions. And for me, and you were familiar enough with the system, we’ll get through it relatively quickly. But for a new user, they might have to take some time. And the other aspect that really does help is that we, fortunately interfered less with our employee’s day, if it’s something that’s self-paced for them, right. So that activity, in addition to the module they completed when they could the homework assignment or the assignment that was required for them to get credit for the entire learning plan. Also self-paced done when they could No, it was an assignment that they did within the OnCore training environment, where they would essentially go through the process of following a patient through a calendar, submitting what it would look like for baseline assessments, adding additional procedures, indicating whether the event was planned or whether it actually occurred, or if it actually was missed, not applicable, etc. All of that comes as a consequence of that end user that feild team member, that’s going to be following the calendar that’s built in system for them. So, it is something that there are pros and cons to that approach, I definitely feel like we hit the mark as far as giving them more practice, which is what we were asked for. But unfortunately, it also meant that they had less time with us. So, they had less time to meet with us. And that’s where we supplemented that with a more meetings with the super user community because we do have our super user group. So, they helped give out the message out to their respective departments, we have super users from almost every single department at the institution. But I the last count I had, we had about 190 super users in our group for the entire institution. They’re very active, very engaging, they’re constantly giving us feedback on whether it was good whether it worked. And in addition to that we had what were called q&a sessions. With some departments, I think of these as office hours where we again get to briefly talk about what phase three was about. And then they had their opportunity to ask any and all questions after we highly recommend that they completed the trainings before coming. But we gave them the opportunity to ask us questions without focusing too much on us giving them training material, we wanted them to, at that point in time, ask us questions. What were they stuck on? What didn’t they understand? How are they going to complete it? So, they’re still a blended approach to it. But it was something that we again, change the strategies to give them more practice for phase three. Now, again, it just went live on May 23rd. So knowing whether this was the best strategy or effective I think it was, we haven’t received too many emails as far as questions or things like that. But for the most part, it went well. A lot of people were approachable about this. I think that’s really the key here is to make sure that you have good communication with the department so that they feel open enough to speak out and talk and ask us questions. And you have to be receptive to that as difficult as it might be. Sometimes when you’re bombarded with a bunch of questions from a lot of people. But gotta put that extra mile, if you will, to make sure that the end user is being assisted in a way that’s helpful for their implementation of their protocols, right? So,

00:35:01.240 –> 00:38:19.170

absolutely. And it’s almost like, I was like the analogy. It’s like having your child now driving a car, you feel like you’ve trained them enough, you’ve given them a little bit of experience, but you never really know. And it’s a couple of days after they’ve been driving on their own. And you’re just like, is everything okay? Like, are we sure, like, I don’t want to have anything like any hiccups or accidents or anything like that any speeding tickets and stuff happen. But you won’t really know until a little bit. So, it’s almost like you’re just right in that initial phase of the training wheels are off, you’ve just let them go off on their own. And you got to figure that out. And to go back to one of the things that you’re touching on too and for anybody that’s listening to this, the financials of OnCore, having worked now in the industry for four and a half years and worked with many other different institutions and hearing some of their tips and tricks. I think a lot of times, we fail to forget how many different technology or software applications that our coordinators and our study staff members have to go through like they went to school really to help treat people to really be that one-on-one liaison to help people get better, really at the end of the day. But they also have to know so many different systems. So not only is it just OnCore, but the EMR, if there’s an EDC system, so electronic data capture, and they have to know all these different systems. So one thing that I’ve recently received talking with a couple of other people in the industry was let them know that they’re still getting paid for their time, because like you said, the thing that triggered that thought was when you said homework, because when we’re in school, like we’re not getting paid, we have to be there, we have to do the homework to get a grade like that as the incentive, that’s the carrot is to let us go on to the next stage. Where with our coordinators, our study staff, and anybody that’s going to be working inside the OnCore system or any application is we are getting paid, we are professionals to do our job. And that’s one big thing to remember. But also with that talking about face frame financials, negotiate for that. you can negotiate for about anything you want to now will you always get it No, but it’s also going back to that car analogy going to a car dealership, hey, I want to upgrade to the leather seats, and I want to have the heated and I want to have the AC, I want to have all of this package, well, maybe you get half of that. But half of that is now available to you because of that. And I think that’s something that having spoken with a number of institutions that they’re trying to really make sure that they get into their formulas, they get into those contract negotiations of that time and effort making sure that you recoup that. Because it is something that’s brand new, it may take a little bit longer, because it’s brand new for everybody involved. And making sure that you can kind of account for that does alleviate some of that time pressure and that financial pressure as well. If I could just ask you one other question right now, through phase one, phase two, and phase three, this is a huge project that ya’ll have taken on at MD Anderson, what would be your biggest takeaway, and I want it to be both good. And thing that you would change like what’s been the best part of this whole experience. And then on the flip side of that, if you could go back in add something or change something to the way you approached phase one that could carry on throughout the rest. What would that be? So we’ll just start with what has been like your greatest takeaway of this experience of going through this giant project.

Chapter 4: Final Takeaways

00:38:19.170 –> 00:40:13.160

Final Takeaways
If I had to say the biggest pro, then this is more of a personal thing. Going through it. For me, the biggest pro was the camaraderie that came around coming together to get this done. I met with so many people, so many individuals that I hadn’t worked with previously that this project forced us to work with and make sure that it was implemented properly. Whether it was people from finance, whether it was the CTMS administrator who I had previously met, but I’d never worked with and now I was forced to work with him for this implementation and getting to know his team a little bit more. And, you know, we actually went to the Onsemble conference together and they gave us an opportunity to learn a little bit more about our personal lives and what we were going through as well. So it was absolutely an amazing opportunity to network on a wide variety of different levels. But ultimately with my team at MD Anderson, which because of the pandemic and because a lot of us were having to transition to a remote work environment, that camaraderie that came as a consequence of having to tackle this big project and working together. That, to me is invaluable. That was such a positive experience by absolutely everyone involved. And I absolutely loved every single bit of it as stressful as it was. And so that would, to me would be the biggest pro because I, I’ve seen how everyone again, came together to get the job done, no matter what the obstacle was, no matter what the challenge was. Everyone push forward, made it happen. And so that’s how we got through so many people in that timeframe because it had to get done. You know, at the end of the day, this is a very important system. The other aspect that was very positive is we had the support of our superiors throughout this entire process.

00:40:13.160 –> 00:40:16.130

That’s huge, having leadership help. Yes.

00:40:16.130 –> 00:41:03.040

Not only from our team and our implementation, but from the leaders out in the departments that we were training and helping them push their own staff to make sure they’re completing trainings. And you’re right, just the way you mentioned, nobody likes change. I just learned this system, or I just started here, and I just had to learn this old system, and now you’re telling me I have to learn a new system and have to go through training again. And it’s not always fun. And I get that it’s it can be challenging, it can be difficult. And when you have a full workload, you have a full clinic, you have patients that are depending on you. And you have to deal with that. All of that, in addition to do all this additional training, I can completely understand how it can be very stressful for a lot of our teams at MD Anderson.

00:41:03.040 –> 00:41:19.040

You have to have that huge level of empathy and understanding for that. what would be one thing, one idea, one aspect that if you could go back to phase one, to add maybe change or to adjust that you could have carried all the way through?

00:41:19.040 –> 00:43:15.190

I would have to say we based on some of the components that we received afterwards is how to find an effective way to communicate with all the teams at the institution. That’s the tricky, trickiest part, I think of this implementation. Not everyone uses the same method of route of getting information. So while some people emails was probably the best other people will receive the clinical research news bulletin that we offer, which is once a month, they will take a moment and read that other people will read the MD Anderson intranet website. And we had to do all these different strategies of trying to get the message out. But even then, there was still some complaints about some people not hearing the message, or some people didn’t know about it. And so I think that’s somewhere where we, while we did a great job, it’s still something that based on some of the feedback we received, they still wanted more. And so we’re still trying to find ways to improve on that. But that’s definitely something that could work in. I’m going to slip in one last one. I know you want me to finish. But it’s just because it caught me by surprise at the conference that no one had figured this out is how to properly assess competency. after the fact. And I asked institution after institution, and none of none of them had fully figured out a way to properly assess the competency after the fact. So we’re doing our own method through quality assurance measures to look at the work afterwards and see who’s potentially might be struggling. And again, it’s still too early to know, because we haven’t finished those quality assurance tests. But it is something that we’re trying to get at. But it is important that every institution that is working on this, take that into account, as well as tried to find a way to measure that proficiency of your end user to make sure that the training that you’ve spent all this time on actually worked. And if there is some deficits, that you’re addressing them accordingly.

00:43:15.190 –> 00:43:57.090

Absolutely. And I know there are some reports, speaking just directly to OnCore like the minimum footprint Details Report. But because of the complexity of clinical trials, that there’s so many different nuances and different areas that it can be very difficult without almost hand checking a lot of that, like you said the QA for all of that information. Well. Thank you so much for joining me today, John. I really really appreciate all of the insights and all of these stats that we’re able to go through and kind of just the whole process of blended learning all the way to, you know, lessons learned. And so with that, I’m just gonna go ahead and wrap up with Jonathan, thank you so much for your time being on here with me. And it’s honestly, it’s really great to catch up with you again, and to see you here.

00:43:57.090 –> 00:44:03.150

Likewise, thank you so much for having me. I’ve had an absolute blast. And hopefully I get to see you at the next conference.

00:44:03.150 –> 00:44:21.070

Absolutely, absolutely. So thank you all for listening. So if you enjoy today’s episode, keep a lookout for Advarra social channels, and on and Advarra.com for our next discussion.

Back to Resources