The difficulty with tribute and cover albums is not so much the albums themselves, but how they are often judged in retrospect; in most cases, people find one of two major problems. The first is that the band is too young or too active for a tribute and is seen as not deserving of one, and the second is that the band is so legendary that the bands contributing the covers are seen as unworthy of producing the covers.
In the case of Black on Black: A Tribute to Black Flag, most criticism will certainly fall into the second category as the reputation and influence of Black Flag is hardly in question. While not the most consistent of acts, when the band was ignited, they were nearly without peer. Compilations like The First Four Years and albums like Damaged are so canonized that the bands who dared to tackle the material were certainly brave.
In the case of Black on Black, it's difficult to find fault with the reverence or the quality of the covers at all. The original version, released in 2003 via Initial Records, was the product of a number of nearly universally respected bands: Converge, Give Up the Ghost (as American Nightmare), Dillinger Escape Plan, Burnt by the Sun, Planes Mistaken for Stars, Coalesce and the Hope Conspiracy. All critically acclaimed and well-respected acts that don't tend to evoke much controversy; you may like or not like their music, but you almost certainly respect them.
In all those cases, though, the material is universally solid. Give Up the Ghost and the Hope Conspiracy provide relentless and vicious remakes, while Converge applies their distinctive sound to the band that must have been a singular influence on them. Planes Mistaken for Stars could be the spiritual, if not the musical heirs of Black Flag, and Dillinger Escape Plan and Burnt by the Sun's technical metal approach certainly compliments Greg Ginn's more esoteric leanings.
The addition of material performed by some of metalcore's leading ambassadors -- Bleeding Through, Most Precious Blood, Zao and Remembering Never -- is bound to be a little more controversial, but their covers are actually quite respectable. Zao is probably as influential to the metalcore movement as anyone, but actually turn in a solid cover even if the vocals are -- as is typical for Zao -- indecipherable. Most Precious Blood sticks to what they do best, performing an almost note-for-note cover of "Rise Above," and Remembering Never performs their version with similar aplomb. Bleeding Through is probably the most controversial, though that has more to do with their image than their sound; playing fairly solid metal with faint hardcore influences, the band turns in a competent albeit monotone cover of the Rollins tour-de-force "My War."
Tribute albums like this one serve two purposes: Either they invite older fans to check out the "new kids" take on their favorite band, or they invite fans of the performers to check out their influences. In the case of Black on Black, both purposes are well-served thanks to a well-chosen set of performers and a great selection of material.