The Effectiveness of Using Eval Bee Application Package [PDF]

Citation preview

THE EFFECTIVENESS OF USING EVALBEE APPLICATION PACKAGE



A Basic Research Proposal to Department of Education (DepEd) Cebu Province



JOEL P. RODRIGUEZ https://joelrodriguez0000.wixsite.com/princeyahwe [email protected] 09617800760 Carpo, Tuba, Dalaguete, Cebu. ROLANDO E. BEDUYA Argao, Dalaguete, Cebu LAURENCE ANTHONY MERCADO 09498105480 Casay, Dalaguete, Cebu. MARGIE A. RODRIGO 09617800760 Poblacion, Dalaguete, Cebu. JOHN FRITZ J. PIZARO 09432444491 Poblacion, Dalaguete, Cebu



I. Introduction and Rationale In the new norm, the Department of Education has created learning deliveries that aim to eliminate the barriers erected by the pandemic. One of these is the modular learning delivery where teachers distribute modules to learners and collect the learner’s answer sheets. The aforementioned learning delivery has created problems in terms of increased number of test papers to check, preparation time in reproduction and sorting as well as an increased susceptibility to committing errors in reliability. In line with this, a Peerless Research Group Study (2016), found out that Optical Mark Recognition (OMR) have been proven to be useful in increasing speed in processing forms like registrations, surveys and even exams. According to a study by, Kubota, Yamamoto, and Morioka (2018) entitled, Promoting ICT Education in Developing Countries, they emphasized that technology is one of the best solutions to emerging problems in education. The innovation of the use of Optical Mark Recognition has charted through different kinds of purposes, as for business, for marketing and for education. Jaspreet Singh (2017) the use of OMR in sheet processing easier, faster, and result oriented. The software has offered splendid processing speed for evaluation of the answer sheets. As it holds the capacity to read thousands of sheets in a minute. the speed of the work is reduced in comparison to the work performed physically. Through the years, Casay National High School has recorded a consistent increase in terms of student population. This growth has produced problems on checking increased number of student exams as it consumes too much time and decreased reliability due to errors committed on checking, producing exam scores and counting frequency of errors. The proponents aim to use eval bee application package for the teachers from the District of Dalaguete II to help them automatically tally each number of items, lessen the time of checking, record student examination scores and collect frequency of error.



The system is designed in such manner that can be operated easily by the users. It is an offline based application that lessens financial burden for the teachers as it does not incur data charges when being used. This application also benefits the teachers’ time management and affect motivation in work performance and production.



II. Literature Review The capability of a single answer sheet being scored by optical and electronic scanning techniques at a higher frequency rate can solve several dilemmas that have been experienced by teachers long before the implementation of the new learning modalities. These dilemmas only came to a highlight due to the overwhelming demand of tasks that has to be accomplished in a limited amount of time which strongly affects the performance of the teachers in terms of readiness, effectiveness and efficiency. The proposed effectiveness of an EvalBee application is designed in such a manner that is easy to manipulate; no special training is required to operate the system and is the most cost-effective method when the number of answer sheets to be evaluated is very large and the need is regular. Exams that are conducted using OMR answer sheet checking system addresses the problems of readiness, effectiveness and efficiency. The generation of test cases/data is one of the most difficult aspects of research. While there have been numerous studies in this area, there is still a need to move toward approaches that can produce test cases/data solely from source code, as many software systems have only the source code and no adequate documentation (Eduardo Rohde Eras et al.). The significance of data management and analysis in educational research cannot be overstated, as it is the foundation on which educational research decisions are made. Research data management (RDM), according to Robson (2002), is gaining a lot of traction in the present day, and rightly so. This proves to be beneficial as education is slowly paving through the integration of optical mark recognition systems as data produced are abundant and the provision of an automatic analysis system can further the way in interpreting results. Denzin and Lincoln (2000) states that it has been argued that computer programs can separate the researcher from the data and thus negatively affect the analysis and perception of



meaning. Despite these criticisms, it was mentioned that there are many good computer programs that can assist researchers in managing large data sets and the processes of filing, storage, coding, and so on. The argument on the separation of the researchers from the data somehow applies on EvalBee application but as it focuses on assessment, where objectivity is of utmost consideration, then it may be deemed somehow as helpful so as to avoid biases when rating answer sheets. According to Victoria Kazz (2017), Manual processing is a difficult and mind-numbing task. Data entry entails converting data from a physical to a digital state, as well as a variety of other procedures. In today's world, where technology is at an all-time high, manual data entry is becoming obsolete as everything is achieved through computers and advanced technology. Through the use cutting-edge technology, it reduces and solves the problems of manual data entry. With EvalBee, manual processing is only applied on the creation of the class which is a one-time process. The succeeding use of this application makes manual data entry obsolete since the system already has a set of data embedded within. It only has to scan the data and then the system will automatically recognize it and transform it from physical to a digital state as stated by Victoria Kazz (2017). Every software development group tests its products, yet delivered software always has defects. Test engineers strive to catch them before the product is released but they always creep in and they often reappear, even with the best manual testing processes. Test Automation software is the best way to increase the effectiveness, efficiency and coverage of your software testing. Manual software testing is performed by a human sitting in front of a computer carefully going through application screens, trying various usage and input combinations, comparing the results to the expected behavior and recording their observations. Manual tests are repeated



often during development cycles for source code changes and other situations like multiple operating environments and hardware configurations. According to Nevon Solutions (2012), an automatic answer checker application that checks and labels written answers in the same way that a person does. This software application is designed to validate subjective answers in an online exam and assign marks to the user after the answer has been checked. The system necessitates that you save the original answer. This service is available to the administrative staff. In the system, the administrator will add questions and subjective responses. These responses are kept in notepad files. When a student takes the test, he is provided with questions to answer and a zone to type his responses. After the user enters his or her answers, the device compares them to the original answers stored in the database and assigns grades accordingly. Evalbee falls under the category of being an automatic answer checker application that validates answers and assign marks to the students’ responses after taking the test. As it performs tasks the same as teachers do, this application can sure help lessen tasks to be undertaken and provide more time for the teachers for lesson preparation. Teachers will also have the convenience of having an immediate frequency if error that provides them feedback on the test material that they have made. It does not take too much time unlike the manual counting of frequency which is prone to errors. This also gives more time for the teachers to focus on identifying what items are too easy and what items are in need of revision because vagueness or level of difficulty that are inappropriate to the level of the students. This study addresses the identified gaps that are not studied as it focuses the effectiveness of the EvalBee Application Package in terms of usability, necessity, practicality and reliability. Effectiveness in terms of usability, necessity, practicality and reliability is measured through the perception of the teachers in the district of Dalaguete II. This study is also anchored on the Technology Acceptance Theory (Venkatesh et al., 2003) where perceived usefulness and perceived ease of use determine an individual's



intention to use a system with intention to use serving as a mediator of actual system use. This implies that if teachers’ perception of the EvalBee application package will prove useful or not this will affect their behavior or attitude when it comes to the actual use of the application.



III. Research Questions This research determines the effectiveness of using EvalBee application package by the teachers of District of Dalaguete II. Specifically, it seeks to answer the following sub-problems: 1. What is the profile of the respondents in terms of? a. Age. b. Subject Taught 2. Is there a significant difference in the change of attitude of the teachers towards the use of EvalBee application? 3. What is the perception of the teachers on the effectiveness of using EvalBee application package? 4. Does age affect their perception towards the use of EvalBee application package? 5. Does the subject taught by the teacher affect their perception towards the use of EvalBee application package? 6. Based on the findings, there will be a proposed training workshops in the use of EvalBee app in an automated checking of exam/modules.



IV. Scope and Limitations This study focuses on the Effectiveness of Using EvalBee Application package of the schools from the District of Dalaguete II. The data collection will be conducted to 165 teachers from the district of Dalaguete II using purposive sampling. This study will not cover problems that are not considered as a measurement of the effectiveness of the EvalBee application package. Each of the respondents will be given a questionnaire to answer. The results of this study will be applicable to the respondents and any teacher who are willing to use EvalBee application package. The EvalBee application package is an open source application which refers to a software program or platform with source code that is readily accessible and which can be modified



or



enhanced



by



anyone. Open



source access



grants



users



of



an application permission to fix broken links, enhance the design, or improve the original code. This application package can help the schools from District of Dalaguete II for an easy way of checking, recording, and arranging the names of each student. The proponents aim to use the effectiveness on an application that automatically recognize the examination paper, name of student, record the result of checked paper, and sort the information alphabetically. EvalBee application can also create and edit examination key. It saves automatically to the system to prevent any losses of data. Then, the detected examination of the student will be matched to the answer key system. Another key factor of this system is that it automatically produces a frequency of error upon checking the answer sheets. This application can scan preferably on multiple choice, true or false, identification and essay type of test. The limitation of this application is that can only run-in smart phone 4.1 version above. This application also has limited access user as it can only be managed by the teacher. Students are not allowed to access the application just only to view the result of their examination. The system is able to process individual papers that undergo scanning and



recognition. It is also unable to conduct simultaneous recognition of the examination papers. This application can scan preferably on different types test but has limited capacity on essay type answers since you have to write specific words in the answer key that can be recognized by the system scanner. This is also a time-limited application in the sense that every examination expires on due date. If the teacher would like to cater to late submissions but the session has already expired, then there is a need for the teacher to repost the examination and set new time limitations in the application.



V. Research Methodology A. Sampling The researcher chose the teachers, from both elementary and secondary, in the District of Dalaguete II for the reason that they will be the one who will utilize the application in performing test checking, score recording and frequency of error collection.



B. Data Collection As stipulated in the questionnaire prepared by the proponents of this research, the respondents will be tasked to fill up the respondent’s profile that includes personal information: age and subject taught. Following the filling up of the profile, the respondents will be tasked to answer a survey questionnaire that will determine the change of attitude of the teachers before the introduction of the application and after the use of the application. Another questionnaire will be given to the respondents that will determine their perception on the effectiveness of the EvalBee in terms of usability, necessity, practicality and reliability. The data from this test will be useful to determine if there is a significant difference between the variables in the profile and the change in attitude.



C. Ethical Issues The proponent first secures a written permission and recommendation from the Public School District Supervisors of the Dalaguete II which will be attached to the letter of request to use the EvalBee application. This approved letter of request will serve as a guarantee of the full support and cooperation of the School Administrators and the teachers. Likewise, a written permission is secured from the District Office to administer the said instruments to acquire the data of the study. The required data are the age and the subject taught since this aspect may change/alter the result of the study. While the names of the respondents are optional and are kept strictly with utmost confidentiality. The proponent will not force the teachers to use the application and make sure that their identity and all the gathered data will be kept strictly confidential.



D. Plan for Data Analysis The researcher followed the following data analysis scheduling: The distribution of the questionnaire would take two (2) weeks and the answering of the questionnaires is allotted one week for all convenience to finish answering the questionnaire. After one week the researcher scheduled for the retrieval of the responses and it is allotted with three (3) days retrieval. Responses are encoded and subjected to statistical treatment. The results generated will be interpreted and be reviewed by person in authority to handle interpretation. The interpretation will give rise to the researcher to plan for appropriate actions in order to achieve the recommended plans for action. The proponent of this research used a quantitative approach in generating the results. The respondents’ profile is used to answer about the significant difference in the change in behavior in terms of age and subject taught. The teachers’ perception on the effectiveness



of EvalBee will be measured through Likert Scaling in terms of usability, necessity, practicality and reliability. The researchers also employed McNemar Change Test to determine the change in behavior of the teachers before and after the introduction of the application. McNemar Change Test



BEFORE



NOT NECESSARY



NECESSARY NOT NECESSARY TOTAL



VI. Timetable/ Gantt Chart



AFTER NECESSARY



TOTAL



Job Planning



Task



Days



The researcher prepares the research design and tools to be utilized in the study.



8



2



Validation of the parts of the instrument designed by the researcher (to be validated by the Schools Division Research Committee (SDRC).



2



3



Documents for approval and recommendation of the title and design will be submitted to the Schools Division Research Committee (SDRC) for signature.



5



Once the title of the action research and its design will be approved by the SDRC, then, said Office will release endorsement to the Regional Office 7 for the BERF Program. Once, approved by the RRC, the researcher will begin the process of the action research study.



5



4



5



The request letter is to be addressed to the school heads for the circulation of the online questionnaire survey link or through a printed survey questionnaire to be filled up by the respondents.



5



6



Hand carried or Online submission of responses from teachers and students in the selected secondary schools of Dalaguete Districts.



15



7



Evaluation of the implementation of the online submission of responses.



2



8



Tabulation of results and treatment of data or responses.



5



9



Presentation (with private link), Analysis, and Interpretation of Data via e-mails or table discussions to the Senior High School teachers and students.



5



10



Writing of the summary of findings, conclusions and recommendations.



2



11



Writing of the Proposed Action Plan / Improvements



2



12



Editing, Proofreading and Binding



2



13



Communication and Endorsement for Presentation (Division and Regional Offices)



Multimedia 5



14



Multimedia Presentation and Feed Backing of Results (Division Level)



1



15



Burning of Compact Discs (backup)



16



Table Discussion with Program Supervisors, PPRD, ASDS and SDS for Policy Improvement then, findings shall be emailed to DepEd Offices



1



TOTAL:



65



VII. Cost Estimates First Tranche Activities



Resources/Materials



Estimated Cost



The researcher will prepare the Researcher uses the laptop research design and tools to be and Internet utilized in the study Purchase of materials for the Action Research Proposal e.g. Researcher bond paper, folders, and fasteners



Php 460.00



Printing of proposals



Researcher



Php 44.00



Transportation to submit action Jeepney and bus fares of research proposal and researcher (back and forth) Php 480.00 questionnaire for approval to the SDRC Claim the approved research title Jeepney and bus fares of and design from the office of the researcher (back and forth) Php 480.00 SDRC, and track the letter request to ask for endorsement to the Regional Office 7 for the BERF Program Claim the approved endorsement Jeepney and bus fares of from SDRC and forward researcher (back and forth) Php 480.00 documents to the RRC Once the action research proposal is approved by the SDRC/RRC in the Regional Office 7, the researcher will communicate with the school heads in Dalaguete Districts for the conduct of the study. Printing and photocopying of research materials.



Jeepney and bus fares of researcher (back and forth) Php 480.00



Printed and photocopied Php 800.00 research materials



Tabulation of results and treatment of data or responses Second Tranche Prepare online presentations, analysis, and interpretation of data Encoding of summary/ findings, conclusions and recommendations Encoding of the Proposed Action Plan/Improvements Editing, Proofreading and Binding Printed and photocopied of of manuscripts research documents



Php 420.00



Forward communication and Jeepney and bus fares of the endorsement for Multimedia researcher (back and forth) Php 480.00 Presentations (Division and Regional Offices) Presentation and feed backing of Pocket WiFi results (Division Level) connection Burning (backup)



of



Compact



for



Internet Php 100.00



Discs Researcher used 4 CDs



Php 92.00



Table Discussion with Program Jeepney and bus fares of Supervisors, PPRD, ASDS and researcher (back and forth) Php 480.00 SDS for Policy Improvement, then, findings shall be emailed to DepEd offices. TOTAL: Php 4,796.00 VII. Plans for Dissemination and Utilization



The empathic results of this research include the conduct of training or workshop on the use of OMR Application package in which aims to: 1. Decrease the time spent in checking test papers/modules. 2. Maximize the recording process of getting the scores. 3. Quarterly implementation of the use OMR application specially on summative tests. 4. Improve teacher’s perspective in the usage in OMR Application. 5. Implement the use of the OMR for District Training Workshop.



VIII. References ● ● ● ● ● ●



DepEd Memo No. 143, s. 2017, National and Regional DepEd ICT Summit DepEd Order No. 78, s. 2010, Guidelines on the Implementation of the DepEd Computerization Program (DCP) DepEd Order No. 62, s. 2011, Adopting the National Indigenous Peoples (IP) Education Policy Framework DepEd Order No. 40, s. 2012, DepEd Child Protection Policy DepEd Order No. 13, s. 2015, Establishment of a Policy Development Process at the Department of Education DepEd Order No. 43, s. 2015, Revised Guidelines for the Basic Education Research Fund (BERF)



● ●



● ● ●



●



●



●



DepEd Order No. 4, s. 2016, Amended to DepEd Order No. 43, s. 2015 DepEd Order No. 35, s. 2016, The Learning Action Cell as a K to 12 Basic Education Program School-Based Continuing Professional Development Strategy for the Improvement of Teaching and Learning DepEd Order No. 39, s. 2016, Adoption of the Basic Education Research Agenda DepEd Order No. 16, s. 2017, Research Management Guidelines Kelsey, Sigrid, (2012). Computer-Mediated Communication: Issues and Approaches in Education: Issues Retrieved from www.books.google.com on May 4, 2017 @ https://books.google.com.ph/books? id=qYXvsNUqPTsC&pg=PA67&dq=Media+naturalness+theory+was+developed+by+Ne d+Kock&hl=en&sa=X&redir_esc=y#v=onepage&q=Media%20naturalness%20theory %20was%20developed%20by%20Ned%20Kock&f=false Holcomb, Lori et. al. (2007). Unlocking the Potential of K-12 Classroom Websites to Enhance Learning. Retrieved from www.books.google.com on October 18, 2017 @ https://search.proquest.com/central/docview/206029946/BDB5F0CFC8A8436CPQ/8? accountid=147155 Issa, Tomayess. (2014). A Model to Build an Intercultural Professional Learning Community in a Latin American Higher Education Network: The “Value Cycle” Using Web 2.0. Retrieved from www.books.google.com on October 18, 2017 @ https://books.google.com.ph/books? id=Tt1GAwAAQBAJ&pg=PA18&dq=Personal+Profiles+of+teachers+in+research&hl=en &sa=X&redir_esc=y#v=onepage&q=Personal%20Profiles%20of%20teachers%20in %20research&f=false Kuang-yun Ting(2007) Teaching English using the internet and multiple intelligences approach. Retrieved from www.books.google.com on October 18, 2017 @ https://books.google.com.ph/books? id=eKLJNSzYzKMC&pg=PA120&dq=questionnaire+on+students+perception+to+w ebsites&hl=en&sa=X&ved=0ahUKEwjY26rN_4XWAhWKu7wKHRbHCP8Q6AEIJD AA#v=onepage&q=questionnaire%20on%20students%20perception%20to %20websites&f=false



ANNEX 1: Research Proposal Application Form and Endorsement of Immediate Supervisor A. Research Information RESEARCH TITLE: “OPTICAL MARK RECOGNITION APPLICATION PACKAGE” SHORT DESCRIPTION OF THE RESEARCH: “OPTICAL MARK RECOGNITION APPLICATION IN TESTPAPER CHECKING” is an action research to assess the accuracy of Teachers in using the eval bee app I checking test papers and a proposed training-workshop for the development of the teachers as innovations to help teachers develop their professional skills in checking Test Papers accurately



RESEARCH CATEGORY (check only one) o National o Regional o Schools Division o District o School (check only one) o Action Research o Basic Research



RESEARCH AGENDA CATEGORY (check only one main research theme) o Teaching and Learning o Child Protection o Human Resource Development o Governance (check up to one cross-cutting theme, if applicable) o DRRM o Gender and Development o Inclusive Education Others (please specify): ___________



FUND SOURCE (e.g. BERF, SEF, others)*



AMOUNT



First Tranche Prepare research design and tools (WiFi) Purchase of materials e.g. papers, folder…



Php 115.00



Printing of proposals



Php 200.00



Tabulation of results and treatment Second Tranche Prepare online presentations, analysis (WiFi) Encoding of summary/ findings, conclusions Encoding of the Proposed Action Plan Editing, Proofreading and Binding



Php 200.00



Forward communication and endorsement



Php 214.00



Presentation and feed backing (WiFi)



Php 50.00



Burning of Compact Discs (backup)



Php 48.00



Table Discussion with Program Supervisors



Php 214.00



TOTAL AMOUNT:



Php 2,415.00



*indicate also if proponent will use personal funds B. Proponent Information Lead Proponent / Individual Proponent LAST NAME: RODRIGUEZ



FIRST NAME: JOEL



MIDDLE NAME: PAGDATO



BIRTHDATE (MM/DD/YYYY) 09/19/1989



sex: MALE



position/designation: SHS Teacher III



region/division/school (whichever is applicable) CASAY NATIONAL HIGH SCHOOL CONTACT NUMBER CONTACT NUMBER 2: 1: 4844 518 09617800760 EDUCATIONAL ATTAINMENT (DEGREE TITLE) enumerate from bachelor’s degree up to doctorate degree



EMAIL ADDRESS: [email protected]



(BSED)Bachelor of Science in Secondary Education Major in TLE Business Technology (With CAR in MAVED) Masters in Vocational Education



Immediate Supervisor’s Conforme We hereby endorse the attached research proposal. We certify that the proponent has the capacity to implement a research study without compromising his office functions.



ROBERTO D. MORAN Name and Signature of Immediate Supervisor Position / Designation: _HEAD TEACHER IV Date: ____________________ EDUARDO M. LUMAYAG Ed. D. Name and Signature of Immediate Supervisor



Position / Designation: __District Supervisor_ Date: _____________________ ____________________________________ Name and Signature of Immediate Supervisor Position / Designation: _____________ Date: _____________________ ____________________________________ Name and Signature of Immediate Supervisor Position / Designation: _____________ Date: _____________________