what happens when a test company gets it wrong

[Don Shepherd]

Kids just don't have a chance anymore. HP Prime delivers the wrong answer to a simple median problem, and now test-maker Pearson apparently did not check its own test answer key very closely. See here.

[Garth Wilson]

There are tons of reasons Common Core needs to the thrown into a common grave with no monument or honor given to it.

[Tim Wessman]

Common Core is just fine and has absolutely nothing wrong with it. The actual standard is perfectly well written and is simply a reflection of good teaching methods.

The problem is that a huge number of companies and those looking to make a quick buck have glommed onto it as a way to sell huge amounts of textbooks, tests, prep material, worksheets, and so on. THOSE are what every interprets and understands as "common core" when in reality they are just trying to make money. The vast majority of those complaining and demonizing it are purely looking at what the corporations trying to sell crap are publishing and throwing it all out as "Common Core is evil".

I've read the entire standard cover to cover and find nothing objectionable. 90-95% of it or so was ALREADY what was being done, and that last little bit is what the GOOD educators were doing.

It is a very interesting read. If you've not done so, I'd recommend reading it all cover to cover and not just looking at those trying to sell something. If you have done so, then apologies and I'd love to discuss the finer points and what you find objectionable.

Quote:Asking a student to understand something means asking a teacher to assess whether the student has understood it. But what does mathematical understanding look like? One hallmark of mathematical understanding is the ability to justify, in a way appropriate to the student’s mathematical maturity, why a particular mathematical statement is true or where a mathematical rule comes from. There is a world of difference between a student who can summon a mnemonic device to expand a product such as (a + b)(x + y) and a student who can explain where the mnemonic comes from. The student who can explain the rule understands the mathematics, and may have a better chance to succeed at a less familiar task such as expanding (a + b + c)(x + y).

That obviously is crap that needs tossing...

[Don Shepherd]

(10-11-2014 11:15 PM)Tim Wessman Wrote:  I've read the entire standard cover to cover and find nothing objectionable.

If you have read the math standard cover to cover, then on page 86 you would have found the definition of median:

Quote:Median. A measure of center in a set of numerical data. The median of a list of
values is the value appearing at the center of a sorted version of the list—or the
mean of the two central values, if the list contains an even number of values.
Example: For the data set {2, 3, 6, 7, 10, 12, 14, 15, 22, 90}, the median is 11.

Regrettably (if I am to believe this thread), the Prime calculator would say the median of that data set is 10, not 11. Every middle school math teacher in the United States would tell you that is wrong. Please tell me this will be fixed.

[Bill (Smithville NJ)]

(10-11-2014 11:15 PM)Tim Wessman Wrote:  

Quote:Asking a student to understand something means asking a teacher to assess whether the student has understood it. But what does mathematical understanding look like? One hallmark of mathematical understanding is the ability to justify, in a way appropriate to the student’s mathematical maturity, why a particular mathematical statement is true or where a mathematical rule comes from. There is a world of difference between a student who can summon a mnemonic device to expand a product such as (a + b)(x + y) and a student who can explain where the mnemonic comes from. The student who can explain the rule understands the mathematics, and may have a better chance to succeed at a less familiar task such as expanding (a + b + c)(x + y).

That obviously is crap that needs tossing...

Tim,

Where are you quoting from? I don't see that quote in a previous message. Is it from the Common Core, or.........

Please give us a reference point to your quotes. Otherwise the quotes don't make any sense.

Thanks,

Bill

[Garth Wilson]

This is not the place to discuss politics, so I'll keep this very brief; but it is a very serious issue. CC is not primarily about standards. It is a political agenda, linked with Obamacare and the NSA, designed to get control of every individual. The goal is to move people from capitalism to socialism to communism. The statement that the goal is communism was on a government website until someone realized that there was stuff there that was supposed to be kept secret and took it down in a panic, but watchdog groups had already taken screenshots. I saw it myself. Parents have no representation, no vote, and no say. To make way for it, Obama gutted the 1974 Family Educational Rights and Privacy Act without going through Congress, even though he has no authority to do that. Planned Parenthood is in charge of much of the literature about sex education that will be in the schools, for kids as young as kindergarten, with graphic illustrations of how to masturbate, for kids as young as 2nd grade. "Common Core" includes "core sex skills"! Do you want your kids exposed to that?? I don't think so.

[patrice]

Here in France, even if things evolve, they are a little different.
Schools are mainly government (Ministère de l'Education)
On the day by day basis, teachers are free to build their own lessons and tests, the books are mainly for help, books are not automatically the lessons.
For national tests (such as Baccalauréat), they are made by government employees.

I see a few differences with US:
- All the tests/exams are made public after the start of the exam.
- Since France is spread all other the earth (Europe, Latin America, Pacific ...), there are different set of questions for different places for the same exam.
- Students can get back its corrected answers after the exam.
- The tradition is "open" questions for exams: In the exam, the student get only the questions, not the candidate answers.

[Don Shepherd]

(10-12-2014 03:24 AM)patrice Wrote:  Here in France, even if things evolve, they are a little different.
Schools are mainly government (Ministère de l'Education)
On the day by day basis, teachers are free to build their own lessons and tests, the books are mainly for help, books are not automatically the lessons.
For national tests (such as Baccalauréat), they are made by government employees.

I see a few differences with US:
- All the tests/exams are made public after the start of the exam.
- Since France is spread all other the earth (Europe, Latin America, Pacific ...), there are different set of questions for different places for the same exam.
- Students can get back its corrected answers after the exam.
- The tradition is "open" questions for exams: In the exam, the student get only the questions, not the candidate answers.

Thanks Patrice, that is interesting.

In the US, there are public schools and private schools. Public schools are administered by each of the 50 states. Each state has an approved curriculum that all public school teachers must follow. All public school teachers must be licensed by the state. In my state (Kentucky), the license you earn defines what grade levels you can teach and what subjects you may teach, but sometimes teachers are allowed to teach outside their area of expertise.

Private school teachers do not have to be licensed by the state, and these teachers are free to choose whatever curriculum they want (with approval of the school headmaster or principal, of course). Teachers in private schools generally earn less money than public school teachers.

Standardized tests are created, sold, and administered by private testing companies. These tests are never released publicly nor are they ever returned to the student afterwards. These companies make huge amounts of money from these tests. A significant part of the school year is used to prepare students for taking these tests, and most teachers think this is not the best use of the students time in school.

[Garth Wilson]

(10-12-2014 04:35 AM)Don Shepherd Wrote:  Private school teachers do not have to be licensed by the state, and these teachers are free to choose whatever curriculum they want (with approval of the school headmaster or principal, of course).

My wife, a credentialed teacher, teaches in a private elementary school which is doubly accredited, by both WASC and ACSI. The public elementary schools in the area have no accreditation at all. The curricula at this private school is chosen by committees of the school staff members. She has been on such committees.

The city we're closest to (where she teaches) had a population of about 100,000 years ago when a school-voucher proposition was on the state ballot. I added up the student enrolments of all the private schools I could find in the city at the time, and it came to over 6,000. The highest achievers I know of though are homeschooled. Homeschooling programs have to meet various state requirements too, and parents use homeschool organizations to help them meet those. My sister-in-law is a supervising teacher for homeschoolers. Our daughter-in-law and her siblings were homeschooled through high school, and when they got to college, they found it easier, as they were way ahead of the public high-school graduates.

[Tim Wessman]

(10-12-2014 12:18 AM)Don Shepherd Wrote:  Regrettably (if I am to believe this thread), the Prime calculator would say the median of that data set is 10, not 11. Every middle school math teacher in the United States would tell you that is wrong. Please tell me this will be fixed.

The statistics applications and everything students will use calculate exactly that way.

The discussion in that thread revolved around the hidden CAS command 'median' which used a different but still perfectly valid mathematical definition. Median is "generally" defined as the way described in the document, but there are at least 3 ways to do it that I am aware of. All of them are mathematically and statistically valid.

Bernard chooses to use that other definition in his CAS, and since HP licenses the CAS and did not develop it internally... sometimes there have been things that go through of which we are unaware and might disagree with or wish would be changed. In that thread though, he indicates that it will be easy for him to resolve that to use a different behavior and so that will be changing to bring that CAS command into agreement with the rest of the system should there be another update put out.

[Don Shepherd]

(10-12-2014 08:40 PM)Tim Wessman Wrote:  

(10-12-2014 12:18 AM)Don Shepherd Wrote:  Regrettably (if I am to believe this thread), the Prime calculator would say the median of that data set is 10, not 11. Every middle school math teacher in the United States would tell you that is wrong. Please tell me this will be fixed.

The statistics applications and everything students will use calculate exactly that way.

The discussion in that thread revolved around the hidden CAS command 'median' which used a different but still perfectly valid mathematical definition. Median is "generally" defined as the way described in the document, but there are at least 3 ways to do it that I am aware of. All of them are mathematically and statistically valid.

Bernard chooses to use that other definition in his CAS, and since HP licenses the CAS and did not develop it internally... sometimes there have been things that go through of which we are unaware and might disagree with or wish would be changed. In that thread though, he indicates that it will be easy for him to resolve that to use a different behavior and so that will be changing to bring that CAS command into agreement with the rest of the system should there be another update put out.

Thanks Tim, but the original poster in that thread said that the unexpected behavior of median wasn't just in CAS mode, but in home mode too. Not having a Prime, I can't verify that. If it is fixed, fine, but someone should verify that it works as expected in home mode.

Don

[Mark Hardman]

(10-12-2014 09:19 PM)Don Shepherd Wrote:  Thanks Tim, but the original poster in that thread said that the unexpected behavior of median wasn't just in CAS mode, but in home mode too. Not having a Prime, I can't verify that. If it is fixed, fine, but someone should verify that it works as expected in home mode.

Don

It doesn't work as expected in Home:

[Tim Wessman]

Yes, that is the CAS command that I am talking about. It doesn't matter where you are running it. The way students are taught, the manuals emphasize, and the UI is designed is to use the stat applications. All the results given in those will match what you are all expecting.

[Don Shepherd]

(10-12-2014 08:40 PM)Tim Wessman Wrote:  but there are at least 3 ways to do it that I am aware of. All of them are mathematically and statistically valid.

Tim, two points.

First, I accept that the change to the CAS calculation of median in the Prime will also be effective in the Home mode, when the next update is released. The larger problem is students who have already purchased a Prime calculator and do not upgrade. If they use their Prime to calculate median they will get the wrong answer ("wrong" as defined by their teacher, undoubtedly).

Second, you cannot say that all 3 methods of calculating median are mathematically and statistically valid: they return 3 different answers for the same data set (I am assuming that the 3 methods you refer to are the "correct" one [mean of the two middle values], the one that returns the smaller of the two middle values, and the one that returns the larger of the two middle values). There cannot be 3 valid values for median of the same data set, that is just basic to any discussion of median.

I have to believe that if Messrs. Hewlett and Packard were around today, they would insist that this be treated like the original bug in the HP-35: fix it and give everyone a new fixed calculator.

Yes, I know that 2014 is not 1972, but a truly responsible company would take the hit and fix this for everyone.

[Han]

(10-13-2014 04:50 PM)Don Shepherd Wrote:  

(10-12-2014 08:40 PM)Tim Wessman Wrote:  but there are at least 3 ways to do it that I am aware of. All of them are mathematically and statistically valid.

Tim, two points.

First, I accept that the change to the CAS calculation of median in the Prime will also be effective in the Home mode, when the next update is released. The larger problem is students who have already purchased a Prime calculator and do not upgrade. If they use their Prime to calculate median they will get the wrong answer ("wrong" as defined by their teacher, undoubtedly).

Second, you cannot say that all 3 methods of calculating median are mathematically and statistically valid: they return 3 different answers for the same data set (I am assuming that the 3 methods you refer to are the "correct" one [mean of the two middle values], the one that returns the smaller of the two middle values, and the one that returns the larger of the two middle values). There cannot be 3 valid values for median of the same data set, that is just basic to any discussion of median.

Here's one def. from wikipedia:

"In statistics and probability theory, the median is the numerical value separating the higher half of a data sample, a population, or a probability distribution, from the lower half."

If the sample is { 1, 5, 7, 9 } then ANY number strictly between 5 and 7 would satisfy this statement.

The (most common) mathematical definition of median is:

Quote:Given order statistics \( Y_1=min_j X_j, Y_2, ..., Y_{N-1}, Y_N=max_j X_j, \) the statistical median of the random sample is defined by
\[ \begin{cases}
Y_{(N+1)/2} & \text{if \(N\) is odd};\\
\frac{1}{2}(Y_{N/2}+Y_{1+N/2}) & \text{if \(N\) is even}
\end{cases} \]

The problem with this definition is that it doesn't give a good intuitive explanation of median. For odd N we pick the "middle" term yet for even N we pick a number that isn't even in the list sample space.

The definition of the term median determines whether the computed value of the median is "right" or "wrong." However, you insist that that mathematics does not allow for multiple interpretations (definitions, if you will) of a particular term. There are reasonable rationales for using the less typical definitions of median. If your domain is the set of integers, then a rational median may seem misplaced. Moreover, in the case of even elements, you are guaranteed that the median is within your set. I am not suggesting that these are the more common interpretations of median, but strictly speaking from a mathematical point of view, there is nothing wrong with "median" being interpreted differently. I do agree, however, that the more common interpretation should have been implemented.

Joke: How does a mathematician catch a lion and place him into a cage? The mathematician goes into the cage, and defines "the inside" as the opposite side of where he is.

[Don Shepherd]

(10-16-2014 05:11 PM)Han Wrote:  If your domain is the set of integers, then a rational median may seem misplaced.

If your domain is the set of integers, then a rational mean may also seem misplaced, but I don't see anyone arguing for an alternate definition of mean, and I think everyone knows that the mean, or average, is a much more common measure of central tendency than median.

Look, I'm not saying that any other definition of the term median is absolutely and positively wrong, but I would argue that if you are going to use any definition other than the standard one that we teach to the kids, you better have a very large footnote explaining that, because it is not what people would expect.

When we teach median to students, we don't say "this is the standard definition, but you should be aware that there may be other possible definitions ...". That would surely open the door for a student to argue a test question marked incorrect because the student has his own definition!

I think HP is aware that this is a goof and they should have caught it during beta testing before it got out into the real world. If one of my students came to me with a Prime calculator that said the median of {2,5,7,9} was 5 instead of 6, I would immediately ban that calculator from being used by my students. I would deem it mathematically untrustworthy.

[lrdheat]

Concerning the lion in the cage, I saw the probabilistic way to catch a lion stated as: Place an open cage in lion country. There is a non-zero probability of a lion being in the cage at time >0. Wait!

[Katie Wasserman]

(Sorry for this way off topic post.)

(10-19-2014 03:28 PM)lrdheat Wrote:  Concerning the lion in the cage, I saw the probabilistic way to catch a lion stated as: Place an open cage in lion country. There is a non-zero probability of a lion being in the cage at time >0. Wait!

Surprisingly, this is exactly how lobsters are caught. Lobsters traps (or pots as they are sometimes called) are really poor ways to catch them as lobsters have no problem getting out of them. So when a fisherman pulls up the trap it's just luck if a lobster is in it. (Of course they do bait the trap to increase this probability.) However, this technique works very well to keep an abundant lobster population off coastal Maine.

(Back on topic.)

I agree with Don, the way to teach math, physics, chemistry or any exact discipline is to teach strict, simple rules then much later in time students can learn that they're not absolute. Didn't we all learn Newtonian mechanics before relativity? A calculator that gives different answers to a basic problem (median) depending on the mode you're in is not a good way for students to learn.

[Les Bell]

(10-19-2014 03:28 PM)lrdheat Wrote:  Concerning the lion in the cage, I saw the probabilistic way to catch a lion stated as: Place an open cage in lion country. There is a non-zero probability of a lion being in the cage at time >0. Wait!

Which reminds me of an old, old humour piece:

Pachydermic Personnel Prediction

by

Peter C. Olsen

A bold new proposal for matching high-technology people and professions.

Over the years, the problem of finding the right person for the right job has consumed thousands of worker-years of research and millions of dollars in funding. This is particularly true for high-technology organizations where talent is scarce and expensive. Recently, however, years of detailed study by the finest minds in the field of psychoindustrial interpersonnel optimization have resulted in the development of a simple and foolproof test to determine the best match between personality and profession. Now, at last, people can be infallibly assigned to the jobs for which they are truly best suited.

The procedure is simple: Each subject is sent to Africa to hunt elephants. The subsequent elephant-hunting behavior is then categorized by comparison to the classification rules outlined below. The subject should be assigned to the general job classification that best matches the observed behavior.

CLASSIFICATION GUIDELINES

Mathematicians hunt elephants by going to Africa, throwing out everything that is not an elephant, and catching one of whatever is left. Experienced mathematicians will attempt to prove the existence of at least one unique elephant before proceeding to step 1 as a subordinate exercise. Professors of mathematics will prove the existence of at least one unique elephant and then leave the detection and capture of an actual elephant as an exercise for their graduate students.

Computer scientists hunt elephants by exercising Algorithm A:
1. Go to Africa.
2. Start at the Cape of Good Hope.
3. Work northward in an orderly manner, traversing the continent alternately east and west.
4. During each traverse pass,
a. Catch each animal seen.
b. Compare each animal caught to a known elephant.
c. Stop when a match is detected.

Experienced computer programmers modify Algorithm A by placing a known elephant in Cairo to ensure that the algorithm will terminate. Assembly language programmers prefer to execute Algorithm A on their hands and knees.

Engineers hunt elephants by going to Africa, catching gray animals at random, and stopping when any one of them weighs within plus or minus 15 percent of any previously observed elephant.

Economists don't hunt elephants, but they believe that if elephants are paid enough, they will hunt themselves.

Statisticians hunt the first animal they see N times and call it an elephant.

Consultants don't hunt elephants, and many have never hunted anything at all, but they can be hired by the hour to advise those people who do. Operations research consultants can also measure the correlation of hat size and bullet color to the efficiency of elephant-hunting strategies, if someone else will only identify the elephants.

Politicians don't hunt elephants, but they will share the elephants you catch with the people who voted for them.

Lawyers don't hunt elephants, but they do follow the herds around arguing about who owns the droppings. Software lawyers will claim that they own an entire herd based on the look and feel of one dropping.

Vice presidents of engineering, research, and development try hard to hunt elephants, but their staffs are designed to prevent it. When the vice president does get to hunt elephants, the staff will try to ensure that all possible elephants are completely prehunted before the vice president sees them. If the vice president does see a nonprehunted elephant, the staff will (1) compliment the vice president's keen eyesight and (2) enlarge itself to prevent any recurrence.

Senior managers set broad elephant-hunting policy based on the assumption that elephants are just like field mice, but with deeper voices.

Quality assurance inspectors ignore the elephants and look for mistakes the other hunters made when they were packing the jeep.

Salespeople don't hunt elephants but spend their time selling elephants they haven't caught, for delivery two days before the season opens. Software salespeople ship the first thing they catch and write up an invoice for an elephant. Hardware salespeople catch rabbits, paint them gray, and sell them as desktop elephants.

VALIDATION

A validation survey was conducted about these rules. Almost all the people surveyed about these rules were valid. A few were invalid, but they expected to recover soon. Based on the survey, a statistical confidence level was determined. Ninety-five percent of the people surveyed have at least 67 percent confidence in statistics.

ACKNOWLEDGMENT

This study has benefited from the suggestions and observations of many people, all of whom would prefer not to be mentioned by name.

[lrdheat]

Les,
Thanks for sharing!