Standard
Historical
Last Updated: Feb 24, 2025
Track Document
ASTM E2263-24
Standard Test Method for Paired Preference Test
Standard Test Method for Paired Preference Test
E2263-24
ASTM|E2263-24|en-US
Standard Test Method for Paired Preference Test
Standard
E2263 Standard Test Method for Paired Preference Test>
new
BOS Vol. 14.03 Committee E18
$
91.00
In stock
ASTM International
Significance and Use
5.1 The paired preference test determines whether or not there is a preference for one product over another product among a specific target population. Knowledge of consumer segments, brand loyalties, the range of product offerings in the marketplace, and the decision risk must be understood when planning a paired preference test.
5.2 The paired preference method is commonly used in tests with one or more of the following three objectives: (1) to establish the preference of a reformulated product vs. a comparison product (the company’s previous formula or a competitor’s product); (2) when there are no prior preference data, to establish the preference of a product vs. a competitor as a benchmark prior to considering a program to improve the competitive status of the product; (3) to establish either unsurpassed or superiority in preference vs. the competition in an advertising claims support test. See Guide E1958 for requirements for unsurpassed or superiority in a sensory claims test. Selected values of Pmax, α, and β will change with all three types of test objectives.
Only (1) establishing a preference for a changed product and (2) establishing preference for a product when there are no prior data will be covered here.
5.2.1 Establish the Preference of a Reformulated Product vs. a Comparison Product: A one-sided test of either NonInferiority or Superiority.
5.2.1.1 One-sided test of Non-Inferiority.If a company reformulates their product, they often want to know if the revised product is as preferred as the original formula. Or, a company might want to test their current product against a revised competitor to establish that their product is not lower in preference. The hypotheses tested are:
| HO: the reformulated product is substantially lower in preference compared to the comparison product. |
| HA: the reformulated product is not substantially lower in preference compared to the comparison product. |
What constitutes a “substantially” lower preference is defined via a so-called non-inferiority margin, that needs to be carefully selected. Under the alternative hypothesis, the reformulated product is hence either at least equally preferred to the comparison product, or it is slightly less preferred, but not by too much.
5.2.1.2 Non-Inferiority Tests are Different from Tests of “Parity”: In these cases, the objective of the test is to establish the non-inferiority (“not worse than”) of one product compared to the other, rather than “parity”. It is best to avoid using the term “parity” in the setting up of the test hypotheses and the planned statistical analysis, since the researcher’s only interest is determining if a loss in preference occurs. The researcher would be quite happy if the revised product is more preferred than the comparison product. The test is one-sided if the action standard is that the product must be as preferred or can be more preferred vs. the comparison product. In turn, “parity” refers to test hypotheses that have both an upper and a lower bound. Non-inferiority testing to establish that a product is not less preferred vs. a comparison product is a test with a single bound. Researchers may discuss testing for “parity” with stakeholders as this word is in common usage. With respect to the specific hypothesis tested, the researcher tests a hypothesis that specifies a single lower boundary, not a hypothesis with two boundaries. Tests for non-inferiority require fewer respondents than tests for parity.
5.2.1.3 First Considerations in a Test of Non-Inferiority: Select a Pmax to represent what you expect an acceptable preference split to be. This is generally lower than 50:50 by a small so-called non-inferiority margin (otherwise it would be a test for superiority, see 5.2.1.4), maybe 45:55, so in slight disfavor of the reformulated product. The implicit assumption made is that a reformulation might still be acceptable to save costs if the potential deterioration of preference is rather small. The null hypothesis here is that the reformulated product is less preferred than the comparison product by more than the specified margin, and the desired outcome is to reject the null hypothesis and hence to conclude that the reformulated product is in fact not (much) less preferred than the comparison product.
The main risk to avoid is to wrongly conclude that your product is less preferred over the comparison product. Thus, a value of α = 0.05 is typically selected. Lower values of alpha can be selected if the researcher wants to be extremely sure that noninferiority is not erroneously found. The desired outcome of this test is to reject the null hypothesis. A reformulated product (A) is at least as preferred as the comparison product (B). The test is one-sided. The value of β should also be low, as the researcher wants to avoid missing to show non-inferiority if that is the case. Beta depends on the true but unknown preference rate for the reformulated product; typically, it is assumed to be 50:50 for this purpose, but it could be any other rate larger than the non-inferiority margin. Selection of the appropriate number of respondents is determined by Pmax, α, and β, One-sided tests require fewer respondents than two-sided tests.
5.2.1.4 One Sided Test for Superiority.Some one-sided tests are conducted to determine if a product is more preferred than a comparison product. The reasoning here is the same as listed above in a one-sided test for non-inferiority, except that the lower bound is now given by 50:50 (that is, preference rates are exactly the same) or potentially a larger value like 55:45 when the objective is to not only show superiority, but also superiority of at least a certain margin.
The hypotheses tested in the most common case of a 50:50 margin are:
| HO: the test product is lower or equal in preference compared to the comparison product. |
| HA: the test product is higher in preference compared to the comparison product. |
Under the alternative hypothesis, the test product is thus more preferred than the comparison product. If a lower bound larger than 50:50 is chosen, under the alternative the test product is preferred by more than this lower bound compared to the comparison product.
An example for testing for superiority might be one where there is a “new to the world” product compared to an existing competitive product. Select a Pmax to represent what you expect a reasonable preference for the product of interest will be. The main risk to avoid is to wrongly conclude your product is preferred over the competitor’s. Alpha values are typically set at 0.05. Lower values of α can be selected if the researcher wants to be very sure that superiority is not erroneously found. The desired outcome of this test is to reject the null hypothesis. The alternative hypothesis is one-sided: The “new to the world” product (A) is preferred over the comparison product (B). The value of β will be determined by the risk of missing a difference. One-sided tests require fewer respondents compared to two-sided tests. The selection of the appropriate number of respondents is determined by Pmax, α, and β, in the test.
5.2.2 When there are no preference data, establish the preference of a product vs. competitor or improve the competitive status of a product Two-Sided Preference Testing: There are scenarios where no consumer preference or acceptance data have been collected by the company. This situation can occur when the product is sold in channels other than retail, or when the company is too small to have budgets for consumer product testing. This test establishes a benchmark preference.
The hypotheses tested are:
| HO: there is no difference in preference between the two products. |
| HA: the product of interest is either less preferred or more preferred vs. the comparison product. |
5.2.2.1 First Considerations in a Two-Sided Test to Establish a Preference Benchmark: Select a Pmax to represent what size of a preference split you want to be able to detect with high probability. The main risk to avoid is running an insensitive test which may result in missing a difference in preference when there is one. The preference difference may either favor, or not favor, the product of interest. Thus, a typical value of alpha is 0.05, but beta values can vary, for example, 0.20, 0.10, or 0.05. There are three possible outcomes of this test: (1) the product of interest is less preferred vs. the comparison product, (2) the product of interest is more preferred vs. the comparison product, or (3) no difference in preference between the two products can be shown (not necessarily indicating that products are similar in preference). If the product of interest is less preferred, the company may decide to initiate a product improvement program, to ensure the sales of that product do not erode. If there is no difference in preference, further testing might be performed to show no difference in preference which the sales team can then use to point out to the fact that on a preference basis the two products are not different. If the product of interest is preferred vs. the comparison product, the company can decide if they want to use this preference information in their sales organization or launch a larger scale claims test to have sensory preference claim. More respondents are needed in a two-sided test compared to a one-sided test. The selection of the appropriate number of respondents is determined by Pmax, α, and β, in the test.
5.3 A test result of superiority or parity does not ensure that the test conclusion is correct. An incorrect test result can be obtained when the sample of respondents is selected in a way that does not reflect the true preference in the population of interest, or when the number of respondents is too small to correctly reflect the preference status of the two products among the target consumer group. Careful selection of Pmax, α, and β and an appropriate selection of respondents is needed to minimize the risk of drawing an incorrect conclusion in forced-choice paired preference testing.
Scope
1.1 This document covers a procedure for determining preference between two products using either a two-alternative forced-choice task, or with the option of choosing no preference. Preference testing is a type of hedonic testing.
1.2 A paired preference test determines whether there is a statistically significant preference between two products for a given population of respondents. The target population must be carefully considered.
1.3 This method establishes preference in a single evaluation context. Replicated tests will not be covered within the scope of this document.
1.4 Paired preference testing can address overall preference or preference for a specified sensory attribute.
1.5 The method does not directly determine the magnitude of preference.
1.6 This method does not address whether or not two samples are perceived as different. Refer to Test Method E2164 for directional difference test.
1.7 A paired preference test is a simple task for respondents, and can be used with populations that have minimal reading or comprehension skills, or both.
1.8 Preference is not an intrinsic attribute of the product, such as hue is, but is a subjective measure relating to respondents' affective or hedonic response. It differs from paired comparison testing which measures objective characteristics of the product. Preference results are always dependent on the population sampled.
1.9 This standard does not purport to address all of the safety problems associated with its use, when testing includes hazardous materials, operations, or equipment. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and to determine the applicability of regulatory limitations prior to use.
1.10 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.