The Schema That Never Gets Written

The short version

You can win a public records request and still lose. You file, you wait out the statutory clock, you survive the exemption fight, and eventually a response arrives: a scanned PDF, or a spreadsheet with columns named FLD_07 and STATUS_CD, and no document anywhere explaining what STATUS_CD = 4 means.

You now hold records you cannot compare to anything. Not to the next county's records, not to the same agency's records from three years ago, not to the forty-nine other states. And a fact that cannot be compared cannot become a pattern — which means it cannot become a finding, a story, or a bill.

The absence of a common schema across public agencies is not a technical gap waiting on funding. It is a policy outcome, and it is produced deliberately, in rooms where lawyers advise agencies on what their recordkeeping should look like. Not by conspiracy — by ordinary, individually defensible legal advice, given over decades, whose aggregate effect is a public record that resists aggregation.

What a schema is, and why it is the whole fight

A schema is the agreement about what the fields are called, what values they may hold, and what those values mean. It is boring. It is also the single thing standing between "here is a box of documents" and "here is what this institution does, at scale, measured."

Consider what changes if fifty state court systems recorded custody dispositions against one agreed field list. A question that today takes a decade, a research grant, and fifty separate legal fights — do similarly situated cases produce systematically different outcomes depending on which county you file in? — becomes a query. One person could run it in an afternoon. That is not an exaggeration of the technology; it is a description of what standardized data does.

Now consider that from the other side of the table. If you advise an institution whose exposure is proportional to how legible its behavior is, a common schema is the worst thing that could happen to your client. Not because your client has done anything unlawful — you may sincerely believe it hasn't — but because legibility invites measurement, measurement invites comparison, and comparison invites a plaintiff, a reporter, or a legislator with a chart.

Where the schema dies

Nobody sends a memo saying "ensure our data cannot be compared to anyone else's." The outcome is assembled out of six ordinary decisions, each of which has a respectable justification, and each of which is made with counsel in the room.

1. Records policy is drafted by lawyers, not by data people

The people who decide what an agency records, in what form, and for how long are almost never the people who will later be asked to produce it. Records management policy, retention schedules, and public-records response procedure are legal work products. Their author's professional obligation runs to limiting the institution's exposure. Producing a legible public record is, at best, not their job — and at worst, in direct tension with it.

2. The retention schedule decides what exists long enough to be asked for

Retention schedules — the approved documents governing how long each category of record is kept before lawful destruction — are the quietest lever in the entire system. A category defined narrowly, or assigned a short retention period, disposes of a question before anyone thinks to ask it. This is entirely lawful. Destroying records on schedule is compliance. It is also why the retention schedule, and every amendment to it, belongs in your request.

3. Procurement never requires a data dictionary

When an agency buys a case management system, the contract specifies uptime, support hours, and price. It very often does not specify that the vendor must deliver documented schema, a defined export format, or bulk extract at no incremental cost. The result is that the data model of a public institution's core records is a private commercial artifact, and the agency's honest answer to "what fields do you have?" is that it would have to ask the vendor — who may charge to answer.

4. "That is not a record" and "we do not create records"

Records law generally obliges an agency to produce records it has; it generally does not oblige an agency to create records it does not have. Sound as far as it goes. Applied to a database, it becomes something else: agencies routinely take the position that running a query is creating a new record, and therefore outside the request — even where the same query runs nightly for internal reporting. The doctrine converts every structured database into a black box that can only be photographed one page at a time.

5. Format as an exemption that never has to be claimed

Releasing structured data as PDF is legally unremarkable and analytically devastating. It launders a database into a picture of a database. Nobody has to assert an exemption, nobody has to defend a withholding, and the requester gets a response that technically satisfies the statute while carrying almost none of the information the statute exists to convey. Always request native electronic format explicitly, and always object in writing when structured data arrives flattened.

6. The standards exist — adoption is optional

This is the part that makes the "we would love to, but it is technically hard" defense collapse. Common exchange standards for exactly this kind of government data already exist and have for years. The National Information Exchange Model grew out of justice-domain data sharing precisely to let agencies exchange records across jurisdictional lines. The National Center for State Courts and the Conference of State Court Administrators have published court data and statistical reporting standards. At the federal level, the OPEN Government Data Act — Title II of the 2018 Evidence Act, signed in January 2019 — requires executive-branch agencies to publish data in machine-readable, open formats by default.

None of that reaches most of the institutions discussed here. State courts are not federal agencies. The judicial branch is carved out of the federal records statute and partially carved out of many state ones. And the standards that do exist are, almost without exception, voluntary. The engineering was finished years ago. What was never supplied was a mandate — and the reason a mandate never arrives is that the professional bodies best positioned to advocate for one are staffed by the same bar that would be measured by it.

The counter-move: request the schema, not just the records

Here is the practical turn, and it is the most useful thing on this page.

The documents describing how an agency stores its data are themselves public records — and they are far harder to withhold than the data they describe. A data dictionary contains no minor's name. An entity relationship diagram identifies no litigant. A retention schedule is, in most states, an affirmatively published document. A vendor contract is a procurement record. The privacy and confidentiality exemptions that shield the underlying data almost never reach the metadata that describes its shape.

Requesting the schema does three things at once:

• It tells you what to ask for next. You cannot write a precise request against a database you have never seen described. One schema request converts every future request from a guess into a citation.
• It destroys the "not maintained in that format" defense in advance. An agency that has given you its field list cannot later claim it does not know what fields it has.
• The refusal is itself the finding. An agency that will not tell the public what shape its records take has said something newsworthy about itself, on the record, in writing. File the refusal. It is evidence.

The schema request clause

Append this to any request built from the public records template, or send it on its own. It is deliberately scoped to records about the recordkeeping system, which is what keeps it out of the usual exemption fight.

SCHEMA AND DATA DICTIONARY REQUEST

In addition to the records requested above, and independently of them,
I request the following records describing HOW the agency stores the
data at issue. These are records about the recordkeeping system, not
the underlying case or client data, and no exemption applicable to the
underlying data applies to them.

1. The data dictionary, record layout, or field definition document for
   each case management system, database, or application used to record
   the matters described above — including field names, data types,
   permitted values, code tables, and the definition of each code.

2. The entity relationship diagram, logical data model, or physical
   schema documentation for each such system.

3. Any published or internal specification describing the format in
   which this agency transmits data to any state or federal body,
   including any NIEM IEPD, XML/JSON schema, flat-file layout, or
   extract specification currently in use.

4. The current, approved records retention schedule applicable to each
   category of record above, together with any amendment to that
   schedule adopted in the past ten years and the memoranda supporting
   each amendment.

5. The list of standard, canned, or scheduled reports the system is
   capable of producing, together with the report definitions or queries
   underlying them.

6. Any contract, statement of work, license, or amendment with the
   vendor of each such system, limited to the provisions governing
   data ownership, data export, format of extracts, and any fee charged
   for producing an extract.

7. Any legal opinion, guidance memorandum, training material, or
   written policy issued to staff regarding the treatment of public
   records requests for data held in these systems, including guidance
   on when data constitutes a "record," when a query constitutes
   "creating a new record," and the standards for asserting exemptions.

8. Any record of this agency's evaluation of, participation in, or
   decision to decline adoption of a common or standardized data
   exchange format for these records.

Regarding item 7: to the extent any responsive document is withheld as
privileged, I request a privilege log identifying each document by date,
author, recipient, general subject matter, and the specific privilege
asserted, as required by [STATE STATUTE].

I request all of the above in their native electronic format.

A model minimum schema

The second half of the argument is that you cannot only demand a standard — you have to be able to put one on the table. "Adopt a common schema" is a slogan. A field list is a proposal. The following is deliberately small: it is the minimum set that makes cross-jurisdiction comparison possible, and every field in it is data the agencies already hold.

Three design choices in that table carry most of the weight, and they are the three that get negotiated away first:

• Controlled vocabularies, not free text. The moment a field accepts arbitrary prose, comparison ends. Most "we publish that already" claims resolve, on inspection, to a free-text field.
• Stable identifiers across releases. If the hash of a matter changes every time an extract is generated, nothing can be followed over time — and longitudinal analysis is the only kind that answers the questions worth asking.
• A logged redaction, not a blank cell. Withholding is legitimate. Withholding invisibly is not: it makes a redacted dataset and an incomplete one look identical, and it lets the size of what is being withheld go permanently unmeasured.

Model statutory language

For use alongside the Family Court Transparency Act, or as a standalone amendment to a state public records act. This is the provision that turns a voluntary standard into a floor.

Common Schema and Machine-Readable Publication. (a) Each agency subject to this section shall publish, and maintain in current form, the data dictionary, field definitions, code tables, and record layout for each system of record used to store the data required to be reported under this section. (b) Data required to be published under this section shall be published in an open, machine-readable format conforming to the common schema adopted under subsection (c), and shall not be published exclusively in a page-image or fixed-layout format. (c) The [ADMINISTRATIVE OFFICE / SECRETARY OF STATE] shall adopt, by rule and after public comment, a common schema for data published under this section, and shall give preference to an existing national standard where one is applicable. (d) Each published extract shall identify the schema version to which it conforms and shall include, for each field withheld or redacted, the statutory basis for the withholding. (e) An agency's assertion that responsive data cannot be produced in the required format shall be accompanied by a written statement, signed by the agency head, identifying the specific technical limitation relied upon and the cost of remedying it.

Subsection (e) is the load-bearing one, and it is the one that will be struck first. It costs an agency nothing to say "our system cannot do that" in an email. It costs considerably more to say it in a signed document that a reporter can request next year.

When they refuse

"The data dictionary is proprietary to our vendor." Ask for the contract provision that makes it so. Then ask how the agency evaluated the bid without it, and how it plans to migrate off the system. A public agency that has contracted away its ability to describe its own records has made a procurement decision the public is entitled to see.

"We have no document that describes the fields." Then ask for what necessarily exists instead: the report definitions, the extract specification used for state or federal reporting, the training materials given to data-entry staff, the screen-layout documentation. An operating database with no documentation anywhere is a finding about internal controls, and one an auditor will care about.

"Producing a schema would require creating a new record." It would not — you have requested existing documentation, not a report. Say so in writing and cite the specific items. This objection is frequently reflexive and frequently withdrawn.

"Those are attorney-client privileged." Applicable, at most, to item 7. Request the privilege log. The existence and volume of legal guidance to staff about how to answer records requests is itself informative, even when the contents are properly withheld.

Silence. The strongest refusal you can receive, because it converts to a constructive denial in most states and puts the burden on the agency. Follow up citing the statutory deadline, and log it.

What to do with what you get

Schema responses are worth pooling in a way that individual data releases are not. Fifty agencies' field lists, side by side, produce something no single request can: a map of where the divergence actually is, which is the difference between "agencies should standardize" and "here are the eleven fields that already appear in forty-three of fifty systems under different names."

That second sentence is a bill. The first one is a wish. If you run one of these requests, tell us what came back — including, and especially, the refusals.

A final reminder

This page is aimed at systems, not at people. Do not use a schema request to investigate the attorneys or officials handling your own matter. The target is the recordkeeping architecture of an institution, and the argument only works while it stays there. The use policy explains why.

On the standards referenced above: the National Information Exchange Model (NIEM), the court data and statistical reporting standards published by the National Center for State Courts and the Conference of State Court Administrators, and the OPEN Government Data Act (Title II of the Foundations for Evidence-Based Policymaking Act of 2018, signed into law January 14, 2019) are described here in general terms as to their existence and voluntary or branch-limited application. Coverage, current status, and applicability to any particular agency vary and change; verify against the current authority before relying on any characterization here in correspondence with an agency.