|
|
|
|
|
| Students Speak Toolkit > II. The Focus Group Blueprint > B. Act > 3. Draw a random sample |
A random sample is a sample obtained in such a way that each element in the population has an equal chance for selection, or each of the possible individuals in the population has an equal chance to be selected. That is, each student in the population has an equal chance of being selected for the focus group.
Drawing a random sample is fun, and it boosts the credibility, validity, and reliability of your research more than any other single step you can take. All that is required is careful attention to procedure. We emphasize that drawing a random sample is definitely not the same thing as picking every third person from a list, or just trying to get a bunch of students with different characteristics together in one group - even if you decide to be extremely fair and include those students you would rather not include.
Your recruiters should allocate about four hours to carry out the work of sampling. (This presumes that one of you has already generated the appropriate numbered lists of students.) Do this work in a private space. Be sure to keep the details of the process confidential in order to protect student anonymity.
Drawing a random sample is a precise process. In the instructions that follow
we describe two different methods for drawing numbers for a random sample. You
may use either one. One involves using a printed random number table. The other
involves using an inexpensive, widely available computerized random number-generating
program. We recommend Mathbook+, a computer program that generates lists
of random numbers according to your exact specifications. You can order this
program from Conrad Button's Software; 20230 Lake Riley Rd.; Arlington, WA 98223
(cost is $20).
Neither method is more complicated than the other, though we emphasize again
that the steps are precise and must be followed closely. You may want to enlist
a statistics instructor at a local university to help with this part of the
process. Be assured, however, that drawing a random sample is easier and more
interesting than learning to program your VCR!
Call us or e-mail us at RKI (859-231-8308 or rki@robertsandkay.com) if any
of these instructions are unclear.
These instructions presume that you are drawing the random samples yourselves. If, however, you are working with a school data person (like the administrator in the Office of Research and Evaluation in Fayette County) to draw the samples, you may be able to skip some of the steps presented here. You will need to work with the school data person to figure out which steps you'll need to do.
Step-by-step instructions for conducting a random sample
We will walk you through the process of drawing a sample of students who will take part in one focus group. It is likely you will need to repeat the process several times, because you will be conducting multiple groups. Each group will have its own set of characteristics, predetermined by the work team. Taken all together, the different groups will represent the students whom the work team believes have the answers to the research questions.
For each student focus group you will be looking for a set of students with specific characteristics (grade, race/ethnic background, average academic performance, etc.), depending upon the decisions made by the work team during the planning phase. This is called a purposive sample, a sample in which you randomly select participants that fulfill a set of predetermined characteristics.
Here's an analogy that may help to explain a purposive sample a little more clearly:
Imagine that your population (your targeted focus group participants) consists of M&M's, and that you need to randomly select ten participants for the focus group out of a huge bowl of mixed plain and peanut candies. You don’t want just any old M&M in the group – a straight random sample. You want your final sample to include a mixture of colors, as well as a mixture of plain and peanuts. Specifically, you want two red plain, one red peanut, one green plain, two green peanut, one orange plain, one orange peanut, one brown plain, and one brown peanut, all drawn in a scientifically random way from the whole bowl. You don’t want any yellow M&M’s. Reach in without looking and draw out an M&M. Make sure it fits one of your slots. Continue to blindly plunge your hand into the bowl until you have an M&M that fits each of the ten descriptions above (two red plain, one orange peanut, etc.). Discard any yellow candies. Simply throw out any duplicate colors/types. That is, if you have already filled the slot for a brown peanut – and you randomly pick another one – throw the second one away (actually, you could eat it). Do not go hunting for a green plain candy if that is the only slot left unfilled – keep blindly drawing and discarding non-green plain candies until you finally draw the one you need. You should end up with a set of ten M&M’s that exactly meets your specifications and that was drawn at random.
We return to the example in which a work team addressed the challenge of discovering how to improve the learning climate in the pre-high school grades. Here, again, are the group characteristics the work team determined:
Information needed: List of all P-3 graders in each school, numbered, including information about their gender, race/ethnic background, academic performance, length of time in the district, and whether they are children of educators in the district.
Information needed: List of all students in grades 4-5 in each school, numbered, including information about their gender, race/ethnic background, academic performance, length of time in district, and whether they are children of educators in the district.
Information needed: List of all middle school females, numbered, including information about their gender, grade level, race/ethnic background, academic performance, participation in school athletic programs or not, length of time in district, and whether they are children of educators in the district.
Information needed: List of all middle school males, numbered, including information about their gender, grade level, race/ethnic background, academic performance, participation in school athletic programs or not, length of time in district, and whether they are children of educators in the district.
So, what do you do now?
Create a "slot sheet."
Create a "slot sheet" for each group, based on the work team's decisions about the specific types of participants each group should contain.
If we apply the M&M analogy to the randomization process for student focus groups, the characteristics change but the basic idea of "slots" for participants remains the same. You need a different slot sheet for each group you are constructing.
It's important to create your slot sheets before you begin the random number process, just so you're clear about what kinds of students you need in the focus groups. Keep in mind that you will be drawing more names than you need, for two purposes: 1.) You will select and recruit a few extra students to make sure the groups will be full in spite of unreturned permission forms or absences, and 2.) You will draw some extra names to use as "reserves" in case you need them later on (you don't need to include the "reserves" in the first round of recruiting).
As a working example, we offer a slot sheet for one of the middle school female groups as described in the example above. Each slot represents a student who offers certain characteristics desired by the work team and who does not represent any of the excluded characteristics.
Obtain the relevant lists of enrolled students; aim for lists that contain as much information about the students as possible (gender, grade, average academic performance, race/ethnic background, etc.).
Divide the lists so that they are manageable/easy to use. For example, to recruit for the middle school groups as described above, you may choose to divide your lists in one of these ways:
Dividing the lists reduces the number of random numbers you will have to draw and increases the chance that you will "hit upon" a student who will fill one of the slots, since the list will already provide for one or more of the targeted characteristics (for example, grade level). If you follow this approach, you will be conducting a random sample of each list. For example, you will draw numbers from a list of 6th grade females until you have filled the slots for 6th graders. Then you'll move on to a list of 7th graders.
Depending on how the school computer generates the student lists, your recruiters may have to do some manual sorting to get the lists into the appropriate format.
You can work with a single list of all students, though this will probably
require more time and effort in the long run, particularly if you are seeking
students who fit a very specific set of characteristics.
Number each list, beginning at the top with 1 and continuing through the end. (If you are using more than one list - for example, one of 6th grade males and one of 6th grade females - start each list with #1.)
How to number the lists if you are using a random number table...
If you are drawing a sample from a list of 99 students or less, use a two-digit numbering system. Start with 01, 02, etc. If you are drawing a sample from a list of 999 students or less, use a three-digit numbering system. Start your numbering with 001, 002, etc. This is to ensure that the number next to the student looks exactly the same as the number on the random number table (i.e. all have the same number of digits).
(If you are using a computer program, you don't need to worry about this.)
Draw a random sample from each list (if working with more than one).
For each focus group to be drawn, you will need four things:
If you are using a random number table...
Example: If you are drawing a sample from a two-digit list of 90 6th grade females, use the two digits on the right of each random number. If you were using the random number above - 565473 - your first randomly drawn student number would be 73.
Example: If you are drawing a sample from a three-digit list of 600 middle school students, you will use the three digits on the right of each random number. Your first randomly drawn student number would be 473.
If the random number table has just the right number of digits for your population, use each number just as it is. If the random number table has too few digits, you cannot use it and must get another table that contains at least the number of digits necessary to represent your population.
After using the first number you selected, continue down that column of random numbers, matching each number with a numbered student name on your list. Write down the name of each student as s/he is drawn into the sample.
When you reach the bottom of a column, move to the top of the next column on the right. If you have gone through the final column on the right side of a page, go to a new page, or, if you only have one page of numbers, go to the first item in the left-hand column and work your way through that column.
If you are using Mathbook+ (or some other computer program)...
Open the Mathbook+ software. The default selections are "Integer Values Only" and "Unsorted Output." Keep these settings.
In the "Upper Bound" box, enter the total number of student names on the list you are working with (in our example above, there were 240 students on the list, so we would enter "240" as the upper bound). The "Lower Bound" will always be 1. The "Quantity of Numbers to be Generated" should be several times larger than the number of students the recruiters must select. We suggest something between 50 and 70 if your total number of students on the immediate list (Upper Bound) is less than 200, and something between 150 and 200 if your Upper Bound is over 1,000. This will give you plenty of numbers in case of duplicates and ineligibility.
Click <OK> and the program will generate a list of numbers. Select <File>, <Save>, name the file, and save it. It will have a file extension of .dat and can be opened in WordPerfect or Word. (When you open the document, the software will ask from what format you are converting the file; the default it has chosen should work fine.)
See the box below for an example of what your random number list should look like.
Example of Randomly Drawn Numbers for Middle School Females
34 8 65 64 108 28 16 115 98 20 69 124 96 26 25 53 121 10 16 97 65 41
Go down the column(s) of random numbers, matching each number with a numbered student name on your list. Write down the name of each student as s/he is drawn into the sample.
Each random number, in order, selects a student with that number. Do not skip a number or toss out a student whose name is drawn, except in the following situations:
Skip a randomly drawn student number if you come to numbers that are higher than your total population. For example, if your total population is 250, all numbers higher than 250 are simply ignored or "tossed out" of the sample. (This shouldn't happen if you are using a computer program.)
Skip a randomly drawn student number if you come to numbers that you have already drawn. Many random number tables or random number lists contain repeat numbers, because each number has an equally likely chance of coming up every time a new number is generated. If you have already drawn student number 73 and the number 73 comes up again, toss it out and move to the next number in the column.
Skip a randomly drawn student number if you draw numbers for students who have been categorically ruled out of the sample, for example, students who moved into the district less than six months ago or students whose parents are educators in the district. (You may not have that information at this time, which is why it's a good idea to draw the names of a few extra students in the sample, just in case you have to exclude some students later on.)
Skip a randomly drawn student number if you draw the name of a student who doesn't fill any of the remaining slots in the sample. Toss out this number and move to the next one.
Example: You are drawing your sample from a numbered list of 240 middle school females. As shown in the "slot sheet", you have decided to invite a total of 12 students in order to account for those students who cannot participate or who do not return permission forms (only 8 will be included in the group). You plan to recruit four 6th graders, four 7th graders, and four 8th graders for the focus group. You have also decided to draw the names of five "reserves," whom you may recruit at a later date if necessary. Altogether, you will be drawing the names of five 6th graders, six 7th graders, and six 8th graders. As your sample proceeds, you begin to fill these categories. For example, you draw numbers 149, 68, 200, and 7, all of whom are 6th graders who represent the necessary characteristics. Your next number is 89, also a 6th grader. Place her in the first "reserve" slot and move on to the next number. Your next number is 32, another 6th grader. Toss her out and move on to the next number.
Most of your samples will be somewhat complex, depending on the characteristics you want to include in the focus group. The principle remains the same, however: Draw the randomly numbered students in the order the numbers appear on the table of random numbers. Toss out all those who duplicate already-filled slots, or who must be excluded for other reasons.
Using the random number list, match the first number on the list to the corresponding student name. Check on that student's characteristics. In the box above, the first number on the list is 34. Jessica Davidson, a 6th grade Caucasian female athlete with a 'B' average, is #34 on the list of students, so we write her name in the slot next to this description.
Note: You may fill in the slot sheet as you draw your random numbers and match them with students' names, or you may wait until you have drawn the names of 20 or so students to begin filling in the slot sheet.
Continue this process until you have filled in a name next to every slot on your slot sheet, including the reserves. Important: Do not skip any numbers on the random number table just to find a good match!! This means that if you draw the numbers of eight 6th grade females with an 'A' average in a row, you simply need to discard any extra names and continue down the list until you draw the number of a student who does fill a needed slot. In order to make this scientific, you must draw the sample on the basis of the exact order in which the numbers were given to you. If one number does not fit one of the categories you are looking for, then go on to the next one. If there is no student who corresponds to a certain number, go on to the next number. (Again, if you are using Mathbook+, this shouldn't happen if you entered the correct "Upper Bound.") If the work team has decided to exclude certain types of students (and this information is provided by the student lists), skip the numbers that correspond to those student names and keep going. Also throw out any duplicates. For example, in the number list in the box above, the number 16 appears twice. Just cross the second one out and keep moving down the list.
By the end of this process, you should have the names of 12 students to invite as participants for a female middle school focus group, along with the names of 5 reserves.
If you divided your lists at the outset according to grade and gender (or some other characteristic), you will need to repeat this entire process for each list. Number each list, generate random numbers for each, and match the numbers with student names. Fill in the slot sheet accordingly.
If recruiters need information about participants that is not provided by the school's student lists - such as whether or not they have parents employed by the school system - you have one more step to complete in the random sample process. After you have drawn all of the names for your focus groups, screen students on the additional factors, either by phone or in person, as determined earlier by the work team. The screening will allow you to determine whether or not some students drawn through the random sample need to be excluded from the focus groups. If you need to exclude some students on the list, return to your slot sheet and move the reserves up to the participant list, in the order in which they appear. Screen these students on the additional factors. Continue as needed until you have a full list of participants who meet all the selection criteria. (Note: If you need to move some of the "reserves" to the participant list at this time, return to your random sample to draw the names of several additional students who will now be added to the slot sheet as "reserves." In other words, if you need to move some of the student names around, make sure that you still end up with 12 recruits and 5 reserves.)
Next: Recruit student participants.
|
|
|
|
|