========================================================================== Team member names: Jisu Kim jisuk@andrew.cmu.edu Jenny (Heewon) Chi hchi@andrew.cmu.edu Sally Cheung sscheung@andrew.cmu.edu Jasmine (Tianjiao) Qi tqi@andrew.cmu.edu Joyner (Qiaona) Yu qyu@andrew.cmu.edu ========================================================================== For Team Assignment I.2, please * Turn in a complete, revised draft of your Team Assignment I.1, covering all the points [(a) through (e) for project 1, and (a) through (f) for project 2] below. Please label these points in your revised draft, so they are easy (for me!) to find and read. * The revised draft should include very carefully thought out information about Target Population (item b below), Sampling Frame (item c below) and Method of Data Collection (item d below). Note under (a) below that some citation of sources for background information on each project proposal is also needed. * Also make any additional revisions required or suggested in my comments below. Please revise ALL PARTS of the proposal for next Tuesday. Even if I have not commented much on one or more points, all aspects of the proposal can be improved. * Assignment I.2 will be graded using the same scheme below, but with higher standards throughout. ========================================================================== Project Proposal 1: How satisfied of our students with CMU dining options. And what potential changes could be made to make them more satisfied. OVERALL: * EVERYONE can do a better job of convincing me why each of their projects is interesting enough to actually bother to do. * If you look at undergrads, grad students, those on meal plans, and those not on meal plans, you will have a lot of work to do. Make sure that you are proposing something that is both interesting and doable within the time you have this semester. (a) Interesting research question? Doable? Each team member provides one piece of previous research or results on this question? * The research question is potentially interesting, and the answers might lead to changes on campus if they were publicized. * Looking at both students on the meal plan and those not on the meal plan will provide a broader view of what "works" and what "doesn't work" about the plans. However, it essentially doubles the work you will need to do, since at least some questions will be different for the two groups: this is like doing two surveys, and you will have to have a big enough sample for each survey. * Since you answered a-e or a-f from the longer project outline, you don't have research/results on this project. Please provide one citation, and at least one sentence on that citation, for every member of your group, in the revised draft. Here are some questions that might lead to interesting citations: When was the last time such a survey was carried out on campus? Does dining services ever do such surveys? If so, how could you produce a more trustworthy result? Are there Tartan stories about satisfaction with the meal plan before the changes made last year? After? Grad students, and faculty & staff, can also participate in meal plans. Do enough of them participate to make it worthwhile to survey them? [Could you shrink to just undergrads? do you need to expand to faculty and staff?] * pls answer carefully for I.2 (b) What population do you want to make inferences about? "The population that we wish to make inferences about is CMU students who dine on campus regularly. It is a subset of the population sampled. Questions that identify this population will be included in the questionnaire." * This is a nice, specific description. From your answer about the sampling frame I gather you are interested in both graduate and undergraduate students. - it is harder to capture graduate students in some sampling frames than others. So if you keep grad students in your target population, you'll need to choose a sampling frame that does capture them - you may be making too many distinctions to effectively carry out your survey, within your buget of one semester of work: o students who regularly dine on campus, vs those that do not o students who partic in meal plan, vs those that do not that's potentially 4 groups. Do the 4 groups get different sets of questions? if so, you will need a big enough sample for each set of questions. This is like doing 4 surveys instead of one. - alternatively you may wish to focus on only one of these groups (e.g. students who partic in meal plan). However, then some questions can't be answered (like, why do students stop participating in the meal plan?). * pls consider carefully these tradeoffs and refine your target population for I.2 (c) What population will you sample? Is this different from (b)? "The population that will be sampled is CMU students in general, both grads and undergrads." * This is too general a statement. If I take it seriously, it would lead to overcoverage (What about students at the Qatar campus? At the Silicon Valley campus?). So you must not mean this seriously. What *do* you mean? * The sampling frame should either be a list of students that you can take a random sample from, or a set of students that are well defined by actions you take (e.g. all students that pass by the fence between times X and Y on days A, B and C -- not necessarily a great fame [undercoverage? selection effects?] but an example of a frame that is not a list). * pls answer carefully for I.2: SAMPLING FRAME (d) How do you plan to carry out the survey & why? "Both through emails and facebook, because they provide greatest potential sample size in terms of CMU students, not to mention the convenience." * Relying on two methods (rather than just one) for contacting students is a good idea, you will capture more students that way. However, it is not enough to send email to "everyone" or to post a generic invitation on facebook to participate in your survey: students who respond to a general invitation are different from those who do not respond; they may have different attitudes about dining as well. * once you have contacted the students, how will they respond? surveymonkey? email answers back to you? meet them for face to face interviews? etc. * whatever the method, nonresponse will be a big issue. - Face to face interviews make for great response rates but would be at least very time consuming to carry out. - if you could get a student phone book, you could conduct a phone survey. this might be effective at getting better response rates but not use up a ton of your time. - if you could get a list of email addresses, you could conduct an email or email-and-web survey. - for email/web surveys nonresponse will be a bigger issue. - for email and web surveys, get only about 25% or less response rate. - how can you improve this with followup and/or prenotification (e.g. postcards)? how many times, and how, will you followup? - who responds, who doesn't? how would this affect biases in your results? - if you are stuck with low response rate, what would you need to know or be able to do, to argue either 1. the sample you got is still like a representative probability sample; or 2. you know what adjustments to make so that the sample can be treated as a representative probability sample * pls consider these issues and refine your MODE OF DATA COLLECTION for your proposal draft for I.2 (e) What variables will you measure? "class year; frequency of dining on campus; what do you like/unlike about the food services (lists provided); overall satisfaction; whether or not they are on mealplan; potential improvements" * This is a sensible start. As you focus your research question more (e.g. is it more about quality of food service or attractiveness of meal plan?), you should develop a more refined and well-defined set of variables to measure. * pls answer carefully for I.2 ========================================================================== Project Proposal 2: How satisfied are university students with Pittsburgh bus service? OVERALL: * EVERYONE can do a better job of convincing me why each of their projects is interesting enough to actually bother to do. (a) Interesting research question? Doable? Each team member provides one piece of previous research or results on this question? "How satisfied of university students with Pittsburgh bus service. How does the Pittsburgh [Port] Authority respond and plan according to those in need of this service exactly in terms of the times in which students take buses the most and whether or not buses arrive on time. While looking into satisfaction of bus services, we hope to see what students like and not like about such bus services and what students are affected most by buses." * Most of these questions can be addressed by survey. However, "how does the port authority plan..." is not a question you can answer with a survey. * No previous research/results on this project (that's ok, you id a-ef from the longer project outline) . Please provide one citation, and at least one sentence on that citation, for every member of your group. E.g.: - have surveys of student satisfaction with PAT service been done recently? - what conclusions have they drawn? - are these conclusions likely to change with your survey? - have there been any changes made (or proposed) that respondents could be expected to voice opinions about? * pls answer carefully for I.2 (b) What population do you want to make inferences about? "We wish to make inferences about students who use bus as their main transportation tool. This population is a subset of the sampling population. Questions that identify this population will be included in the questionnaire." * you can be a little more specific: - university students only? - are there specific universities you are concerned with, or just "all" students? - etc. * pls answer carefully for I.2: TARGET POPULATION (c) What population will you sample? Is this different from (b)? "Major university students in Pittsburgh will be the population sampled." * more specificity needed: - which universities will you visit? - is there a list of students (email, phone #, etc.) on each campus? - you have specified for (d) "face to face - convenience / accidental". Who would you see, or not see by accidental meetings on the street - or at a bus stop? What biases could this introduce into your survey? - could you specify a set of criteria for what kinds of people you would approach on the street to reduce some of these biases? ("ensuring the quality of data collected") - is "using the bus as their main transportation tool" the first question you will ask? those who say no are excused from the survey? * please specify a workable alternative way to select students into your sample, that addresses some of the problems with "convenience / accidental" selection. * pls answer carefully for I.2: SAMPLING FRAME (d) How do you plan to carry out the survey & why? "Face to face (Convenience/accidental sample) as major method, because we would like to randomly approach students in local area while ensuring the quality of data collected." * some of my reactions above point to some issues you should address with "man on the street" face to face interviews * whatever the method, nonresponse will be a big issue. - Face to face interviews make for great response rates but would be at least very time consuming to carry out. [but who would you miss - what kind of selection effects would there be - with man-on-the-street interviews?] - if you could get a student phone book from each campus, you could conduct a phone survey. this might be effective at getting better response rates but not use up a ton of your time. - if you could get a list of email addresses for each campus, you could conduct an email or email-and-web survey. - for email/web surveys nonresponse will be a bigger issue. - for email and web surveys, get only about 25% or less response rate. - how can you improve this with followup and/or prenotification (e.g. postcards)? how many times, and how, will you followup? - whatever the method: who responds, who doesn't? how would this affect biases in your results? - what would you need to know or be able to do, to argue either 1. the sample you got is still like a representative probability sample; or 2. you know what adjustments to make so that the sample can be treated as a representative probability sample * pls answer carefully for I.2: MODE OF DATA COLLECTION (e) What variables will you measure? "when do you take the bus the most; what's the most important factors (frequency; convenience; comfortable...); why do you like/unlike the bus service, and why" * This is a good start, and it will get more specific as you think more carefully about this proposal * in addition, what about demographic or background variables? e.g. male/female, income, where do they live, etc. -- things that might affect how necessary bus service is for that person. * pls answer carefully for I.2 (f) Sample questions for off-campus project? * these questions are an ok start, but mostly they ask "do you like it" questions. If respondents do not, what action could someone take? Please revise and add to your questions, to get at more things that someone could do something about. Right now only question 5 addresses this sort of issue. * think about the order of the questions. E.g., shouldn't 5 come right after 3? * we will talk about question formats later in the course. this will help you refine your questions. * pls answer carefully for I.2 ================================================================== Total grade: Project Proposal 1: (out of 48; 8 pts/part) Project Proposal 2: (out of 56; 8 pts/part) ----------------------- Total 104 NOTES: On I.1, everyone gets 100 points, regardless of the comments I made above. On I.2, I will grade each part about out of 8 pts/part. This come out to a bit more than 100 pts, which will just be extra credit for a team that really nails I.2. Each person on the team gets the team score recorded in the gradebook. If your team gets 84 pts on I.2, then each team member gets 84 pts in the gradebook. ==================================================================