Making changes to a live study

What to Do When a Study has been Soft-Coded

What should you do when a study has been soft coded (data locations not specified) and you are ready to go live and need to make sure that changes made during the interviewing process do not greatly affect the data layout?

There are basically three different approaches. The one you will want to use will depend on how many changes and what kind of changes you expect to make after the study has gone live. On a job that will run for only two or three nights, you might want to use approach #2. For a wave study, you might want to use approach #3.

Approach #1

You expect some amount of changes. This is the standard recommended method.

Issues

Positives
- The data position can be determined for any existing question from the spec file.
- Existing data questions can easily be modified because default values have all been filled in.
- You are less likely to make a data error.
Negatives
- Repeats are expanded, so the file is much larger and repeat blocks are difficult to modify.
- Default values are all filled in, so each question has unnecessary information in it.

When Going Live

Make sure the top of your program has the HARDCODE compiler command in it right after the Header statement and before your first question block.

>PURGESAME
~PREPARE COMPILE

[EXAMPLE]
{!HARDCODE }
Rest of actual questionnaire

~END

Then, after you have done the final compile, but before you go live, follow these six steps:

1) Rename your existing spec file <study>.qpx to something like <study>.qpxorg
2) Rename the existing check file <study>.chk to something like <study>.chkold
3) Rename your existing qsp file <study>.qsp to <study>.qpx.
4) Edit the file <study>.qpx and make the following modifications:

Add the following lines at the top of the file.

~Comment
Any comments you would like in the file.

>PURGESAME
~PREPARE COMPILE

Add the following line at the bottom of the file.
~End

5) Make sure that questions in this file have data locations on the label line.

Example:

'44.50  404
{ QGENDER:  404
Enter Gender of Respondent
!FIELD
1 Male
2 Female }
It is the 404 on the line with QGENDER that is needed.

6) Re-compile with the new <study>.qpx file.

Making Changes After Going Live

Adding Questions

If you are adding questions, look at the CHK file and find the last column that was used. Make sure that you put any new questions in UNIQUE data positions after that point in the file. For instance, if the CHK file says the last column used is 575, put the new question in data position 601. This would look something like the following:

{ QNEW: 601
This is an added question
!NUMERIC,,,0-100 }

This puts the question called QNEW in position 601, so it neither shifts nor overwrites any existing data. Be sure to double check the CHK file to make sure the new column(s) assigned do NOT overwrite any existing columns.

Removing Questions

If you wish to remove a question from the survey, use the HIDE option on the label line.

{ DONTWANT: 273 HIDE
Don’t want this question anymore
!VARIABLE }

Modifying an Existing Question

If you need to modify an existing question and are not increasing the number of data columns it uses, then there is no problem.

If you need to increase the size of an existing question, you need to both create a new question and hide the original one. You must copy the original question, assign the new question a new data position outside of the existing data, use the HIDE option on the original question and change its label, so as not to create a duplicate label.

Two unfortunate consequences of this change will be:

The data for this changed question will need to be netted back together at some point.
Previously suspended interviews will probably not FIXRESUM.

If there are questions you think are likely to increase in size, then you can put “.n” on the label line of the question in the original file, where “n” is the number of data columns to allocate for that question.

Important: After you have recompiled with your changes, look at the new CHK file and make sure you have no unwanted data overwrites and compare <study>.CHK with <study>.CHK.ORG to make sure all the old questions are being assigned the same data positions.

NOTE: After you have recompiled with your changes, look at the new CHK file and make sure you have no unwanted data overwrites and compare <study>.CHK with <study>.CHK.ORG to make sure all the old questions are being assigned the same data positions.

Approach #2

This is probably best for minimal changes.

Issues

Positives
- You are able to use your original spec file.
- Repeats are not expanded.
Negatives
- You cannot determine where the data is stored from your spec file.
- Data overwrite errors are more likely to occur.

When Going Live

After doing the final compile, but before going live, you will want to follow the next four steps:

1) Rename the check file <study>.CHK to something like <study>.CHK.ORG

2) Look at the CHK file to determine what the last data column was that was used.

3) If you are not currently using the work area, then set the WORKSTART option in the Header to a column greater than that column. For instance, if the last column used is 575, then set WorkStart=601.

[EXAMPLE,32000,"Example Header",TextStart=2001,WorkSstart=601]

4) Recompile the spec file

Making Changes After Going Live

Adding Questions

If you are adding questions, just hard code the new question in the work area using the WORK keyword, or place it into a UNIQUE previously unused data column anywhere after the WORK_START column. If WORK_START is 601 and the work area has not been used yet, you can put the first new question starting in data position 601.

For example:

{ QNEW: WORK
This is an added question
!NUMERIC,,,0-100 }

{ QNEW:  601
This is an added question
!NUMERIC,,,0-100 }

In both cases, this will put the question called QNEW in position 601, so it neither shifts nor overwrites any existing data. Be sure to double check the CHK file to make sure the column(s) you assign do NOT overwrite any existing columns.

If you wish to be able to resume old suspends, make sure that all new questions/compiler directives have a label name.

Removing Questions

If you wish to remove a question from the survey, use the HIDE option on the question label line.

{ DONTWANT: HIDE
Don’t want this question anymore
!VARIABLE }

Modifying an Existing Question

If you need to modify an existing question without changing the number of data columns it uses, then there is no problem.

If you want to reduce the number of columns the question uses, you need to put “.n” on the label line, so that the question will continue to use “n” number of columns. For example, if the following question had its maximum changed from 100 to 50 you would want to put .3 on the label line so it would continue to use the 3 columns, instead of 2.

{ CHANGE: .3
Enter in your number here
!NUMERIC,,,0-50,,DK }

If you want to increase the size of an existing question, you need to both create a new question and hide the original one. You must copy the original question and assign the new question a new data position past the WORK_START position and use the HIDE option on the original question and change its label, so as not to create a duplicate label.

Two unfortunate consequences of this change will be:

The data for this changed question will need to be netted back together at some point.
Previously suspended interviews will probably not FIXRESUM.

NOTE: If the .CHK files check out, then compare the MAKENAMES setting in the old QSP file with the new one. If they are the same, you should be okay. If the new one does not match, then existing suspend files maybe lost. If the current one is higher, that means you likely added in a question without a label and you’ll need to locate that question and put a label on it. Comparing QSP files is probably the best way to do this. If the current makenames is smaller, then you must have removed an unlabelled question or compiler directive. DO NOT DELETE anything. Either use the HIDE option or change the old item to a empty GOTO.

Approach #3

This is probably best for Wave studies where you expect massive changes.

Issues

Positives
- You are able to use your original spec file.
- Repeats are not expanded.
- You can determine where the data is stored from your spec file.
Negatives
- It requires extra labor to assign the data columns before you go live.
- Data overwrite errors are more likely.

When Going Live

After doing the final compile before going live, you will want to do the following:

Copy the existing spec file <study>.QPX to something like <study>.QPX.ORG
Rename the existing check file <study>.CHK to something like <study>.CHK.ORG
Look at the sum file <study>.SUM to determine what data columns the qfile is currently using.
Start modifying the spec file so that those data locations are hard coded into the file. This is a relatively long error-prone process, but it does mean that all previous modifications of the file avoided the use of data positions.
Put the option DATA_LOCATION_REQUIRED in your Header statement.
Recompile with the new spec file.
Make sure that the new check file <study>.CHK exactly matches the old one <study>.CHK.ORG

Making Changes after Going Live

Adding Questions

If you are adding questions, just hard code the new question in a UNIQUE previously unused data column anywhere in the data record.

Removing Questions

If you wish to remove a question from the survey, use the HIDE option on the question label line.

{ DONTWANT: 421 HIDE
Don’t want this question anymore
!VARIABLE }

Modifying an Existing Question

If you need to modify an existing question and are not increasing the number of data columns it uses, then there is no problem.

If you need to increase the size of an existing question you need to both create a new question and hide the original one. You must copy the original question and assign the new question a new data position outside the existing data. Also use the HIDE option on the original question and change its label, so as not to create a duplicate label.

Two unfortunate consequences of this change will be:

The data for this changed question will need to be netted back together at some point.
Previously suspended interviews will probably not FIXRESUM.

Changing to a New QFF File While a Study is Active

You can only use this approach if the changes you are making do not add any references to new quota names that did not exist in the original. If you are adding quota names, you must bring the study down to make those changes.

Copy the current QPX and the live QUO file into a temporary directory/group.
Copy/build a FON/FNX file if you need it for testing.
Use the QFFFILE=<newname> option in the Header of the spec file to cause the new QFF file to be created with the new name.
1. You would usually use a name that is similar to the original study code with an additional number or letter appended to the end of the name.
2. For instance, if the original Study name was BANK, you might have the new name be BANK1.
3. Do not change the study name on the header.
Compile your QPX file and test all your changes.
Once you are satisfied with the changes, then copy the new QFF file into the QFF directory.
1. Remember to also copy the QPX file back into a/the permanent directory.
The Supervisors should then be able to issue a CHI <station number> <NewQFFname> to any stations that are currently working on that study.
1. As interviewers finish the current interview they are on, they will be switched to the updated QFF file.
Once all the interviewers have been switched, then the old QFF file should be deleted, so that the supervisors do not inadvertently start up interviewers on the old QFF file.
1. Do not shut down the study.
2. A supervisor could still issue a Start command to the old QFF, but the interviewer will get a fatal error message when they try to start up.

Note: You may or may not want to rename the new QFF file name back to the original name overnight. This depends on your best practices.

QUICK REVIEW LIST

How to keep from losing existing suspended interviews

Rule 1:

NEVER remove anything. You can hide questions if you do not want them to be asked or us IF logic to skip around them. This includes internal questions and compiler directives as well.

Rule 2:

NEVER change the label, location, or type of a question. If you need to do this, hide the original and make a copy of it and change the copy.

Rule 3:

When adding anything, it is important that the temp names (CFM#####) assigned to unlabeled questions remain the same in order for the old suspends to be able to resume properly. In order to do this, you need to do one of the following:

a) Use the QSP file from the original spec and make all your changes in it. This is your best bet, but also is not very optimal if you have a job that is running for more than a few days as making changes in the QSP file is much more cumbersome that in your qpx. file.

b) If you don’t use the QSP, then start by saving the old QSP file. Then put a label on EVERYTHING you add including GOTO statements, GENERATE questions, and compiler directives. After you have made all your changes and recompiled, you will want to compare the MAKENAMES= setting in the header between your new QSP file with that of the old QSP file. If they do NOT match, then you added something without a label and you likely will have suspend/resume issues. You can compare the old and new QSP files to see where the CFM##### names got out of alignment to know where you added a question without a label.

After you have made all the above changes and believe they are correct, you can test your changes by running Fixresum in “test” mode and it will find any suspend files that are no longer compatible with the existing spec file. We also recommend that you do not run Fixresum in “live” mode unless you have good reasons for doing so.

NOTE: If you believe you have followed the above rules and are still having issues, then we would need to see a copy of the old qpx, the new qpx, and the suspend file.