There are occasions in research where it becomes necessary to weight data in order to reduce as much bias as possible and to be able to generalize findings to a specific population. In telephone survey research, design weights are often employed to reduce bias correcting for differences in the probability of selection due to non-response and non-coverage errors. By taking into account the number of landline phones, the number of adults in each household, the number of available records, and the number of records selected within each geographic strata and density strata these errors can be drastically reduced.
It may be imperative in many instances to match a sample with the population from which it was drawn on specific demographic variables. For example, VIP Research and Evaluation conducts Behavioral Risk Factor Surveys (BRFS) as part of their Community Health Needs Assessments (CHNA). We employ the same BRFS data weighting process that the CDC and many states employ in order to be able to generalize results to the population of specific region based on several demographic variables. Prior to 2011, this approach included using post-stratification weighting – sometimes referred to as cell weighting – which adjusted primarily for age, gender, race, and geographic regions between the sample and the population.
With the rise in the proportion of cell phone-only households and the lack of state-level demographic characteristics of this group, the CDC began employing a more sophisticated weighting method known as iterative proportional fitting, or more commonly known as raking. Because raking considers each of the weighting variables separately, there is less likelihood that categories of age and/or race, for example, would be collapsed than under previous weighting methods.
Raking weighting incorporates the known characteristics of the population into the sample. This is done in an iterative process, with each demographic factor introduced in a sequence. The sequence of factors may be multiple times before the sample is found to accurately represent the population on all factors under consideration.
Some of the advantages of raking over post-stratification weighting are:
• In addition to age, gender, race and ethnicity, and region, the process allows for the introduction of more demographic variables, such as education level, marital status, and home ownership.
• Further, variables such as race and ethnicity can be included in more detail much easier than would be possible using cell weighting.
• More importantly, raking allows for inclusion of the variable telephone source (landline vs. cellular), which is critical in telephone research today.
• Less knowledge is required of the population and one does not need to know detailed demographic data such as the breakdown of age, gender, and race, for example. Because of this, it is suitable for small samples.
• There is less variance in a raked set of weights – the design effect implicit in a raked set of weights will be smaller compared to the design effect implicit in a set of weights obtained by cell weighting.
In the past, the process of raking a dataset was tedious and time-consuming as it required numerous manual iterations to get the weighted variables to closely match those in the target population. However, with the advancement of statistical survey software, raking data is far easier today than it was in the past. For example, using SPSS’s raked weight procedure, researchers can rake an entire dataset on several variables in a matter of minutes. More impressively, all the variables entered into the procedure will closely match those in the target population.
If you would like more information on raking a dataset, feel free to reach out to us anytime at hill@vipreval.com or 847-920-5493.
References
Fricker, Ron and Anderson, Lew. (2015). Raking: An Important Often Overlooked Survey Analysis Tool. Phalanx. 36-42.
Graham, Kalton and Flores, Cervantes, Israel. (2003). Weighting Methods. Journal of Official Statistics, 19:81-97.
Data Weighting – Raking vs. Post-Stratification Weights
There are occasions in research where it becomes necessary to weight data in order to reduce as much bias as possible and to be able to generalize findings to a specific population. In telephone survey research, design weights are often employed to reduce bias correcting for differences in the probability of selection due to non-response and non-coverage errors. By taking into account the number of landline phones, the number of adults in each household, the number of available records, and the number of records selected within each geographic strata and density strata these errors can be drastically reduced.
It may be imperative in many instances to match a sample with the population from which it was drawn on specific demographic variables. For example, VIP Research and Evaluation conducts Behavioral Risk Factor Surveys (BRFS) as part of their Community Health Needs Assessments (CHNA). We employ the same BRFS data weighting process that the CDC and many states employ in order to be able to generalize results to the population of specific region based on several demographic variables. Prior to 2011, this approach included using post-stratification weighting – sometimes referred to as cell weighting – which adjusted primarily for age, gender, race, and geographic regions between the sample and the population.
With the rise in the proportion of cell phone-only households and the lack of state-level demographic characteristics of this group, the CDC began employing a more sophisticated weighting method known as iterative proportional fitting, or more commonly known as raking. Because raking considers each of the weighting variables separately, there is less likelihood that categories of age and/or race, for example, would be collapsed than under previous weighting methods.
Raking weighting incorporates the known characteristics of the population into the sample. This is done in an iterative process, with each demographic factor introduced in a sequence. The sequence of factors may be multiple times before the sample is found to accurately represent the population on all factors under consideration.
Some of the advantages of raking over post-stratification weighting are:
• In addition to age, gender, race and ethnicity, and region, the process allows for the introduction of more demographic variables, such as education level, marital status, and home ownership.
• Further, variables such as race and ethnicity can be included in more detail much easier than would be possible using cell weighting.
• More importantly, raking allows for inclusion of the variable telephone source (landline vs. cellular), which is critical in telephone research today.
• Less knowledge is required of the population and one does not need to know detailed demographic data such as the breakdown of age, gender, and race, for example. Because of this, it is suitable for small samples.
• There is less variance in a raked set of weights – the design effect implicit in a raked set of weights will be smaller compared to the design effect implicit in a set of weights obtained by cell weighting.
In the past, the process of raking a dataset was tedious and time-consuming as it required numerous manual iterations to get the weighted variables to closely match those in the target population. However, with the advancement of statistical survey software, raking data is far easier today than it was in the past. For example, using SPSS’s raked weight procedure, researchers can rake an entire dataset on several variables in a matter of minutes. More impressively, all the variables entered into the procedure will closely match those in the target population.
If you would like more information on raking a dataset, feel free to reach out to us anytime at hill@vipreval.com or 847-920-5493.
References
Fricker, Ron and Anderson, Lew. (2015). Raking: An Important Often Overlooked Survey Analysis Tool. Phalanx. 36-42.
Graham, Kalton and Flores, Cervantes, Israel. (2003). Weighting Methods. Journal of Official Statistics, 19:81-97.
www.cdc.gov/brfss
Latest Posts
Data Weighting – Raking vs. Post-Stratification Weights
November 7, 2018VIP Research and Evaluation Launches New Website
September 4, 2018Archives
Calendar
Categories