Often, marketers want to create several variations on content for a particular URL in the interest of observing which one converts the best.  It is typically an ongoing optimization process, and many different variations may be served over time to that end.  This is called split testing.

The problem with split testing is that it cannot be considered in a vacuum.  First:

1. Page A may perform better than page B with regard to conversions, but:
2. Page A may perform more poorly in the SERPs.

Obviously, this can complicate matters, as it introduces another factor into the performance equation.  For example, if a page converts twice as well, but doesn't rank at all, it is a net loss.  Therefore, search engine optimization principles must be considered when making any changes.

Ideally, all changes would be purely aesthetic; and in most cases doing so would not affect the rank of the page.  If they are not, it creates a complication for this reason, as changing on-page content can clearly influence rankings; but there is another reason, too.  Keep reading.

One method employed to collect data for split testing is to randomly show page A or page B and track conversion rates for each.  Unfortunately, done incorrectly, this practice can confuse search engines or raise red flags.  This is the other problem with split testing.  At worst, implementing this will be perceived as spamming and/or cloaking.  Let's examine the 3 different ways to split test.

1. Redirects directed to the variations randomly
2. Internal program logic displaying the variations randomly
3. Temporal split testing

Methods 1 and 2 are similar in that they randomly display variation A or B.  However, redirects are not ideal in this situation, and they can be avoided.  Therefore, I recommend using internal program logic.  This is consistent with Matt Cutts' recommendation in his video (http://video.google.com/videoplay?docid=1156145545372854697). That means some light programming --

For example, if you have 5 versions of a web page, page[1..5].php,

$id = rand(1,5);
include('page' . $id . '.php');
// tracking program application code here.

The problem regardless is that if the pages are significantly different and served randomly, it might actually be perceived as cloaking.  Matt Cutts hinted at that in the aforementioned video.

Cloaking may be used to show only one version to a particular search engine.  This eliminates the problem whereby a certain version ranks better in search engines than others.  It also eliminates the possibility that it will be perceived as spam academically (so long as you're not detected!).  Yes, cloaking is being used to prevent the perception of cloaking!  However, I'd recommend doing this with a caution that Google frowns upon it.

Either way, if you're detected, you might be sent to the corner.  For more information on cloaking, click here.

The last method, "temporal split testing," is also safe, and extremely easy to implement. Simply collect data for one timespan for A (perhaps a week), and again for B.  However, doing so may be less accurate and requires more time to make determinations.

So, in summary:

0. Don't ignore the organic, possibly dertrimental, effects of split testing.
1. Use internal program logic or temporal-based split testing.  Do not use redirects.
2. You may use cloaking to show only one version to search engines, but Google frowns upon this approach.