How we research

We are trying to establish what is true about an organization, not to catch it doing something wrong. When an organization is exceptional, we say so with the same care we bring to concerns.

A worthy mission is not evidence of effectiveness. Nonprofit status is not evidence of good management. Emotional storytelling is not evidence of anything. We start from the documents and work outward.

We separate four things that most charity ratings blur: what the documents say, what we think the documents mean, how much independent evidence supports the organization's claims, and what nobody can currently determine. Unknown is a legitimate answer and we report it as one.

A value is verified only when we have read it in a primary source: the filing itself, the audit itself, a court docket, a record obtained from a government body. A figure copied from a charity database is not verified until we have checked it against the filing. A figure that appears only in an annual report or on a website is reported.

Each verified fact retains its source, the reporting period, the date we retrieved the document, and where in the document the value appears. When we perform a calculation, such as excluding joint-cost allocation from a program ratio, we say so in a note attached to the fact.

Confidence describes how much weight an assessment can bear. It is assigned to impact claims individually and to the organization's file as a whole.

Three related things are kept distinct. A fact's verification status (verified, reported, unable to verify, contradicted) says whether a single value was confirmed in a primary source. A finding's evidence grade (verified primary source, corroborated, independently evaluated, self-reported, inferred, conflicting evidence, unable to verify, unknown) says what kind of evidence an interpretation rests on, which may combine several facts. Evidence confidence, below, is the file-level judgment of how much weight the assessment as a whole can bear. A file can be high confidence overall and still contain a finding graded unable to verify; both are shown.

We do not publish a single numeric score. Numeric scores imply a precision the underlying evidence rarely supports, and they let a strong financial picture hide a weak impact picture, or the reverse. Instead each organization receives one of five overall assessments, supported by four dimension assessments (financial stewardship, impact evidence, governance, transparency), an evidence confidence level and the findings that produced them. Each dimension states the basis for its grade in a sentence, so a reader can see what was examined and disagree with us on the record.

Questionable is not an accusation of wrongdoing. It means the record contains financial, governance or fundraising practices that a reasonable donor would want to understand and might decide against. Insufficiently documented means we cannot form a view, and we think that itself is important information. An organization handling significant money that cannot show where it went has a problem, even if the money went somewhere good.

When sources conflict, we prefer, in order: court and regulatory records; audited financial statements; IRS filings; government grant and public records; independent evaluations with published methods; peer-reviewed research; the organization's own reports and correspondence; the organization's website and marketing; news coverage. Marketing language is a source of claims, never a source of facts.

Every file ends with a numbered list of every source used, with retrieval dates. Bracketed citations throughout the file refer to that list. If we could not obtain a document, the file says so.

We do not accept payment from organizations we review. We do not sell scores, seals or placements. We do not remove findings at an organization's request; we correct facts when shown to be wrong. We do not psychoanalyze leadership, speculate about motives or report rumors. We do not assign scores to things the evidence does not support. We do not treat high overhead as proof of waste or low overhead as proof of effectiveness; both are inputs, not conclusions.

We do not review organizations in secret. Every published file is public and every organization is notified when its file is published or materially revised. An organization may ask to be examined. It cannot pay for a favorable assessment, alter a conclusion, suppress a finding or buy placement, and it does not control the file that results.

Files change when the evidence changes. New filings, new evaluations, regulatory action and evidence submitted by the organization or by readers all trigger review. The version number and last-reviewed date on every file tell you how current it is. Corrections to published facts are logged and visible. See the corrections policy for how to submit one.