SAN FRANCISCO (MarketWatch) — Burton Malkiel and Charley Ellis are interested in your money.

They don’t want your money. In fact, these two respected investing experts want you to spend less on the mutual funds and other retail products that Wall Street peddles to Main Street investors.

Malkiel is author of the investing classic A Random Walk Down Wall Street, while Ellis penned the pioneering Winning the Loser’s Game. And together, they’ve written a tidy little treatise called The Elements of Investing.

Both Malkiel and Ellis are vocal proponents of indexing as the best, most-appropriate portfolio strategy for individual investors. You’ve come in contact with their work if you’ve ever bought an index fund or an exchange-traded fund, or follow a patient, long-term investment approach.

Malkiel and Ellis also are advisers to a new venture geared to helping investors manage individual retirement accounts. Rebalance is the brainchild of Mitch Tuchman, founder and CEO of investment manager MarketRiders, which constructs low-cost portfolios for do-it-yourself investors. (Tuchman is an occasional columnist for MarketWatch, the publisher of this report.)

In a recent telephone interview, Malkiel and Ellis spoke about how to maximize yield, safety and capital gains during what they expect will be a lengthy period of below-average total returns from stocks.

Rule No. 1: Costs matter

Lower your investment costs and treat the market as an ally, not an enemy, Malkiel and Ellis encourage. This proactive approach makes it more likely that you’ll have more funds for a financially comfortable retirement — which is what Malkiel and Ellis really want to see.

This line of thinking is crucial, they contend, since we’re currently in a challenging investment environment.

It’s highly likely that U.S. stocks will deliver below-average total returns over the coming decade, with the 10-year Treasury yield — the “risk-free” return — so low, Malkiel and Ellis say.

Tack on four percentage points — the historical “equity-risk premium” over the Treasury benchmark 10_YEAR -3.66% — and Malkiel and Ellis are looking at no more than around a 6% annualized 10-year return from stocks, including dividends and before inflation. Read archived story: Why stocks will beat bonds over the next 20 years.

Investment charges can take a big chunk of that gain — for example, a 1.5% annual management fee isn’t uncommon, but it represents 25% of a 6% take.

“Cost, particularly in an era of low returns, is tremendously important,” Malkiel says.

Moreover, a low-cost, diversified basket of stocks stands a good chance of beating inflation, so don’t get too fancy. Be careful about investing in high-risk ventures in an attempt to boost income or capital gains. Says Ellis: “We’re not entitled to a high rate of return and we shouldn’t be reaching for a high rate of return. That’s a sucker bet.”

Rule No. 2: Rebalance your portfolio regularly

Rebalancing is the one form of market timing that passes the test. In this way, you buy out-of-favor assets when they’re cheap and book profits in appreciated holdings. This discipline also keeps you from chasing what’s hot and selling what’s not.

“Individuals take money out of the market at exactly the wrong time, and put money in at the wrong time,” Malkiel says.

“In very volatile markets such as we have had and such as I suspect we are likely to have in the future,” he adds, “there’s just an enormous advantage in rebalancing because it always reduces risk and in very volatile markets will actually increase returns.”

Nowadays Malkiel and Ellis are bearish on Treasurys and point income-seeking buyers to government debt from emerging markets, such as Brazil, where budgets and fiscal policies are relatively strong compared to many developed markets.

The two also recommend that yield-seeking investors lighten up on bonds and generate income through shares of companies that are growing dividend payouts. Says Ellis: “Our situation is new; our thinking must be new.”

Rule No. 3: Indexing wins the ‘loser’s game’

“Winning the Loser’s Game” was Ellis’s explosive essay, originally published in 1975 and later developed into a book, which bluntly told professional money managers that it was increasingly difficult for them to outperform the Standard & Poor’s 500 SPX -0.06% and other benchmark indexes.

Ellis, then a managing partner at investment-industry consulting firm Greenwich Associates, noted that the pros made buy and sell decisions with greater knowledge and speed — and fewer errors. At such a high level, competitors win not because they’re necessarily better, but because others fare worse — a “loser’s game.”

Accordingly, to be an above-average investor, you must keep the ball in play more often than your rivals.

If you can’t, Ellis counsels, consider index funds.

Almost four decades after Ellis cracked its veneer, the investment business attracts an even higher caliber of talent, with correspondingly greater skills and access to information and data. This is a market of hard-nosed institutions and tough traders — no place for dabblers and duffers, Ellis says. Perhaps not coincidentally, low-cost, market-matching index products — especially ETFs — have become more popular in the wake of the 2008-09 market meltdown that sunk professionals and amateurs alike, and Ellis sees this as progress.

“You’re up against the Williams sisters almost every time,” Ellis says, likening Wall Street’s best and brightest to tennis champions Serena and Venus Williams. “I don’t think it makes any sense for individuals to take them on.”

Jonathan Burton is a MarketWatch editor and columnist based in San Francisco. Follow him on Twitter @MKTWBurton.