As published on Medium

Weiss Ratings are on the receiving end of severe crypto community backlash after releasing their Weiss Cryptocurrency Ratings. It’s an ambitious task, for sure, to attempt to rate cryptocurrencies as current conditions make it akin to rating penny stocks — anything can happen. However, it is also a dangerous task. Not for them, of course, but for the consumers, investors, and institutions that will pay for their “research” and use it to inform their investment decisions.

We at Parea agree with the community’s assessment of these score’s viability. What follows is a point by point takedown of the Weiss Ratings philosophy, methodology, and organization.

1. Trust in ratings is broken

Human beings make many of their decisions upon formal and informal ratings and we have an implicit level of trust in them. Whether it’s our friends’ recommendations for activities, Yelp’s restaurant scoring, or Moody’s financial instruments, there’s an expectation that the rating party has our best interest at heart and wants to provide sound advice. However, we now know that this is rarely the case due to perverse systems of institutional incentives. This is usually not a big deal in our immediate lives, but has extreme impacts on our underlying societal structure. Our friends might have bad taste and Yelp is one giant scam to extort restaurants’ money. That’s not ideal, but lives don’t depend on it. However, it gets scary when you realize that rating agencies like Moody’s, S&P, and Fitch, which assess the credit worthiness of the majority of the world’s wealth, are also deeply flawed.

It boggles our minds that it’s already been a decade since the Great Recession. However, although our memories are short, let’s not forget the immense debacle trust in ratings caused during the financial crisis. Credit rating agencies were tasked with analyzing and rating mortgage backed securities, collateralized debt obligations, and other securities created by pooling mortgages together and slicing them up in tranches based on their repayment priority (i.e., the top tranche gets paid out first). Unfortunately, they not only did not fully understand the security structures and underlying assets, but also committed massive fraud, issuing many more investment grade triple-A ratings than were deserved. This was enormously damaging for two reasons:

First, a large section of the debt securities market, such as money market and pension funds, were restricted in their bylaws to only buy the safest, triple-A rated securities. These ratings provided a false sense of security, leading to massive losses for those who could least afford them (e.g., retirees).

Second, investors place a certain amount of faith in these ratings and diminish the amount of research they do proportionally to how highly the security is rated. These assumptions can, and did, backfire.

The credit rating agencies job was to unearth the mortgage issuer’s activities and unmask the machinations of securitizing banks, but they broke trust by doing neither. As a result S&P and Moody’s were sued by federal and state governments, paying fines of $1.4 billion and $864 million, respectively. As investors now flock to cryptoassets, Weiss Ratings is repeating history by providing erroneous ratings of assets and technologies that they do not understand and that can bring significant harm to individuals and institutions relying on them.

2. Broken and opaque “proprietary” methodology

Weiss Ratings claim they have a 5 level process to arrive at a letter grade, but after evaluating what little information is provided about their methodology, we found that level 1 is where they collect a bunch of information and levels 2–5 is them pulling formulas out of a hat until the data says what they want it to say. There is very little mention of what this information is, how they source it, how it is validated, or how their “expert” opinions influenced the ranking.

They have four indices that comprise the letter grade: Risk, Reward, Technology, and Fundamental.

Risk and Reward are essentially analyses of price movements, trends, volatility, etc. Things that tell you what is happening, but not why it is happening. These are traditional tools for financial analytics and we feel Weiss Ratings can accurately assess these characteristics despite their lack of familiarity with this market and asset class.

Where it really breaks down for us is in the Technology and Fundamental indices. They claim to manually analyze whitepapers, public discussion forums, network security, public acceptance, and apparently many other factors, but we dispute these claims based on the following rationale:

First, we were unable to find public sources to establish their credibility and blockchain expertise, including that of their two experts, Italo Silveira and Juan M. Villaverde. They have also claimed to have hired blockchain experts and consultants, but none of their names are public, and while they may exist, we are concerned that they are masquerading as experts for Weiss Ratings just like Weiss Ratings is masquerading as experts to the public.

Second, these technologies are constantly evolving. Only 14 employees list Weiss Ratings as their employer on LinkedIn, of which three seem like they do the actual analysis (an analyst and two senior analysts). We are doubtful that such a small team was able to analyze 70+ cryptocurrencies without their analysis going stale because by the time you do the 20th analysis, the first 10 are already out of date. Perhaps it helped that they analyzed things like Dogecoin, which was abandoned by its creator and hasn’t had a software update in 2 years, but which somehow got the same score as Monero and beat out Bitcoin Cash? They claim it is because Dogecoin has greater usage than Bitcoin Cash, but it feels like the joke is lost on them that this is a coin that isn’t actively being developed or updated, held aloft by a bubble of insanity and good humor.

Third, in order to create accurate ratings, you have to be able to define success and compare like against like. The first problem is that no one knows what success looks like. Bitcoin is the closest example that we have, but even it is facing lots of challenges and uncertainty as scaling poses a major challenge that we are in the early stages of addressing (i.e., Lightning Network). The second is that these aren’t just assets; they are technologies, and are trying to do different things. Monero isn’t trying to be Ethereum — they have vastly different goals and therefore vastly different definitions of what success looks like. Creating a rating methodology that aims to pit them against each other will ultimately fail as it cannot accurately measure their success in their respective fields.

3. Disregard for our philosophical underpinning

“However, decades of experience in the financial marketplace indicate that, once armed with the specific formulas or processes of a ratings model, some rated entities seek to game the system: They try to manipulate data they can influence or control with the goal of achieving an unfair advantage.” - Source: Weiss Ratings

This is what we are fighting against. We are building trustless systems and protocols in public where anyone can see what’s going on and get involved. We know Weiss Ratings is doing this as a money grab, because if they were serious, they would have put in the work to develop a solution in line with our ethos by either making their ratings radically transparent or perhaps even developing a blockchain-based solution with the right incentive structure to accurately rate and report on blockchain project developments. We’re not experts in ratings, but we firmly believe there is a better way to do things — we just have to find it. We hold each other accountable so perhaps it could be as simple as publically shaming teams found to be gaming the system, an area where Weiss Ratings could add significant value by serving as a watchdog.

4. The insane ratings speak for themselves

We won’t delve deeply into this as it was thoroughly covered in various publications (e.g., Op-Ed: The Weiss Cryptocurrency Ratings Are Laughably Bad) and earlier in this article, but we are still stupefied. Something like EOS, which is certainly promising, receiving a higher score than Bitcoin is a clear sign of faulty judging criteria. Dan Larimer has a proven track record with two successful products, Steem and BitShares, but EOS has only been around for 7 months. Bitcoin is a proven product with enormous adoption, huge community support, incredible resilience to internal and external attacks, and a superstar development team. We would argue that it is the best & safest investment one could make in the cryptocurrency space, with Ethereum being a close second due to its great platform, incredible amount of development, and strong team.

This isn’t meant as a knock against EOS or any other cryptocurrency on their list, but these results just don’t pass the smell test. This is particularly concerning because there can only be two reasons for why these ratings were released: they didn’t know how faulty they were (showing their lack of crypto expertise) or they still think they are correct (showing their confidence despite widespread criticism)

“ … which leads me to one of the biggest sources of misunderstanding about our ratings: We do analyze them as INVESTMENTS.” — Source: Weiss Ratings.

Weiss Ratings claims the crypto community is the one that misunderstood, but we believe their focus on price, not value, is apparent to everyone but themselves.

Conclusion

Parea recommends all individual and institutional investors avoid Weiss Ratings’ misguided effort to extract money for, at best, erroneous information. We continue to advise all parties entering this space to focus on the underlying technologies and use cases when making investment decisions. We will do our best to help by publishing analyses of different cryptocurrencies, consensus protocols, governance structures, and other topics, but ultimately, the responsibility rests with you to make sound investment decisions.

Credit

• Ben Rodriguez

Errata

We are only human and we make mistakes. We will include any types of revisions or lapses in logic when found or brought up to us in a dedicated section at the end of each of our essays. In addition, if you have any suggestions for future essays such as formatting, or topics you’d like to see written about discussed, drop us an email at hello@parea.io or leave a comment.