content_category: Risk Management Process

category_confidence: high

specificity_confidence: high

reasoning: Annotator A correctly applied the codebook. After filtering out NOT-list items, zero IS-list facts remain: 'ERM program' is a generic program name without a unique identifier (compare to IS example 'Cyber Incident Response Plan (CIRP)'); 'Atleos' is a company self-reference; 'Board of Directors' and 'executive leadership' are generic governance terms. Annotators B and C erroneously counted the ERM program as firm-specific without recognizing it falls under the NOT list for generic program names. No named frameworks, cybersecurity-specific roles, dates, numbers, or third-party firms appear.

provider: z-ai


reasoning: All annotators correctly identified Risk Management Process as the category. For specificity, Annotators A and B correctly applied the codebook: after filtering, zero IS-list facts remain. 'Cyber Incident Response Plan' lacks a unique identifier (compare to IS example 'Cyber Incident Response Plan (CIRP)' with distinguishing abbreviation), 'Cheniere' is a company self-reference, 'Board' is generic governance, 'third parties' are unnamed entities, and penetration testing/training/assessments are common practices—all on the NOT list. Annotator C erroneously counted the generic program name as a specific fact without recognizing it lacks the required unique identifier.


reasoning: The paragraph describes internal incident response processes, correctly classified as Risk Management Process. For specificity, I find no IS-list facts after filtering: 'CIRT' and 'Cyber Security Watch Team' are team names, not named internal programs with unique identifiers (the IS example is 'Cyber Incident Response Plan (CIRP)'—a program, not a team). For teams, the IS list specifies team COMPOSITION ('Legal, Compliance, and Finance') as the qualifying fact, not team names. 'Incident Response Team' is analogous to the NOT-listed 'incident response plan'—a generic organizational structure. Annotators A and C incorrectly counted team names as specific facts; Annotator B correctly applied the codebook.


content_category: Board Governance

reasoning: The paragraph's primary focus is board oversight of cybersecurity risks, with the first sentence establishing the board's oversight role as the main topic. Sentences 2-3 describe the briefing cadence and mechanism by which the board receives information—management is mentioned only generically (not a specific role like CISO/CTO/CIO) and solely in the context of supporting board oversight. Annotators A and C correctly applied the codebook; Annotator B erred by treating management's reporting activity as the primary focus when it serves the board oversight function, and the Management Role category requires specific role identification (CISO, CTO, CIO) rather than generic 'management.' No IS-list facts are present—'board of directors,' 'management,' 'periodic,' and 'cybersecurity program' all appear on the NOT list, confirming Generic Boilerplate.


category_confidence: medium

reasoning: The paragraph's dominant communicative purpose is describing the company's information security program—its framework adoption (NIST CSF, PCI DSS), Security Operations Center, and incident response exercises. The CIO and CISO are grammatical subjects but serve primarily to attribute oversight of these processes. Applying the disambiguation test: removing role mentions leaves a coherent process description, indicating Risk Management Process. For specificity, Annotator A incorrectly counted named roles (CIO, CISO) toward the QV threshold—per the disambiguation rules, named roles trigger Firm-Specific but do NOT count as QV facts. Only one QV-eligible fact exists ('over 38-years of industry experience'), falling short of the two-fact threshold. Annotators B and C correctly rated Firm-Specific; Annotator C correctly identified the category.


reasoning: The paragraph's dominant communicative purpose is describing internal cybersecurity processes (application design, backup systems, monitoring, anti-malware, training, audits, response team). The Head of IT department mention appears only in the final two sentences as attribution for who oversees these processes—not as the primary focus. Applying the disambiguation test: removing the role mention leaves a coherent process description, confirming RMP. For specificity, 'Head of IT department' is explicitly on the NOT list per the adjudication rules, and no other IS-list facts exist (no named frameworks, cybersecurity-specific roles like CISO/CTO/CIO, specific dates, named firms, or quantified facts). Annotator B correctly applied both category and specificity rules; Annotators A and C erred by counting 'Head of IT department' as a specific fact when it is generic IT leadership, not a cybersecurity-specific title.


content_category: Management Role

reasoning: Annotators B and C are correct. The disambiguation rules explicitly list 'Years of experience' as QV-eligible facts, and this paragraph contains two such quantified facts: 35 years of experience and 24 years in the role. Annotator A erred by not recognizing that years of experience count toward the QV threshold. The role title 'Senior Operations Officer' is not cybersecurity-specific (unlike CISO, CTO, or VP of IT/Security), so it does not contribute to specificity, but the two quantified years of experience independently meet the 2-fact threshold for Quantified-Verifiable.


content_category: Third-Party Risk

reasoning: Annotator A correctly applied the codebook. The disambiguation rules explicitly list 'Third-Party Risk Management Program' on the NOT list as a generic program name without a unique identifier. Annotators B and C made the identifiable error of counting this NOT-list item as a firm-specific fact. After filtering out all NOT-list items (company self-references 'Dime' and 'The Bank', generic program name, generic cadence 'annual'), zero IS-list facts remain, and no recognized standards are named.


content_category: None/Other

reasoning: Annotator A applied the codebook correctly. The disambiguation rules explicitly state that 'generic IT-dependence language ("our IT systems are important to operations")' falls under None/Other. This paragraph contains only recognition statements ('We recognize the importance of...'), IT-dependence assertions ('Our business depends on the proper functioning...'), and generic risk acknowledgment—without describing any actual cybersecurity processes, governance structures, materiality assessments, or strategic decisions. Annotator B erred by interpreting 'recognize the importance of developing' as describing actual processes, when it merely acknowledges importance without describing what the company does. Annotator C erred by categorizing this as Strategy Integration when there is no discussion of materiality, financial impact, insurance, budget, or business implications—only generic risk recognition.


reasoning: All annotators correctly identified Risk Management Process as the category—the paragraph describes internal cybersecurity oversight and processes. For specificity, Annotators B and C correctly applied Generic Boilerplate. Annotator A erred by likely counting 'Software Engineer' or 'IT Team' as specific facts, but neither appears on the IS list. The IS list specifies cybersecurity-specific titles (CISO, CTO, CIO, VP of IT/Security, Information Security Officer, Director of IT Security)—'Software Engineer' is a generic technical role. 'IT Team' and 'cybersecurity risk management program' are generic team/program names without unique identifiers. No named frameworks, specific dates, named products/firms, headcounts, or certification claims appear. After filtering NOT-list items, zero IS-list facts remain.


reasoning: The Audit and Risk Committee is the grammatical subject and the paragraph describes board-level oversight of cybersecurity risks, clearly fitting Board Governance. For specificity, the 'Audit and Risk Committee' is a standard governance committee variant—the codebook explicitly lists 'Audit Committee' as NOT firm-specific, and adding 'and Risk' is a common structural variation that does not create a non-generic committee. No other IS-list facts appear: no cybersecurity-specific titles (CISO, CTO, CIO), no specific dates, no named third parties, no named tools, no headcounts, no certification claims. After filtering NOT-list items, zero IS-list facts remain. Annotators A and B correctly applied the codebook; Annotator C erroneously counted 'Audit and Risk Committee' as a named non-generic committee, which is the #1 error identified in the disambiguation rules (counting NOT-list items as specific facts).


reasoning: Annotator A is correct. The paragraph contains two QV-eligible facts per the disambiguation rules: (1) 'January, 2022' is a specific date, and (2) 'over 20 years' is explicitly listed as a QV-eligible fact ('years of experience: 17 years, over 20 years (these are specific numbers)'). Annotators B and C correctly identified these as specific facts but failed to count them toward the QV threshold, likely overlooking that years of experience qualifies under the disambiguation rules. The category is clearly Management Role as the CDIO is the grammatical subject throughout.


reasoning: The paragraph focuses centrally on vendor qualification and assessment processes, satisfying Third-Party Risk. For specificity, after filtering NOT-list items: 'Risk Dependency Assessment' and 'pre-engagement process' are generic program names without unique identifiers (compare to IS-list example 'Cyber Incident Response Plan (CIRP)' which has a distinguishing abbreviation); 'Critical Vendors' and 'Tier Level' are common vendor management concepts; 'the Bank' is a company self-reference. No IS-list facts remain—no cybersecurity-specific titles, specific dates, named third-party firms, named products, specific numbers, or certification claims. Annotators A and B correctly applied the codebook; Annotator C provided no specificity reasoning and appears to have counted generic process names as specific facts.


reasoning: The paragraph's primary focus is board/committee oversight: it opens with the audit committee assisting the Board in overseeing the cybersecurity program, describes both receiving regular reports, and concludes with incident escalation to the audit committee. The management roles (ISO, CISO) are mentioned only incidentally as part of the reporting chain TO the board/committee, not as the primary subject. Annotator C correctly applied the codebook; Annotator A mischaracterized governance oversight as operational process, and Annotator B treated incidental role mentions as the primary content when they serve only to describe the escalation path to the audit committee.


reasoning: I agree with Annotators B and C. The paragraph describes the company's internal processes for risk identification, assessment, monitoring, and mitigation—core Risk Management Process content. Annotator A erred by treating the grammatical subject alone as determinative, but the disambiguation rules explicitly state that a role mentioned at the beginning followed by process description makes the role incidental. Applying the test: removing 'Management is responsible for' leaves a coherent process description about identifying, assessing, monitoring, and mitigating cybersecurity risks. Additionally, 'Management' is a generic term (on the NOT list), not a specific cybersecurity role like CISO or CTO. No IS-list facts exist—no named roles, frameworks, dates, or verifiable details—confirming Generic Boilerplate specificity.


reasoning: Annotators A and C correctly applied the codebook. After filtering, all items in this paragraph fall on the NOT list: 'Audit Committee' and 'Board' are explicitly listed as generic governance; 'quarterly' and 'annual' are generic cadences without exact dates; 'executive leadership team' and 'Internal Audit team' are standard organizational functions. Annotator B erroneously counted 'cybersecurity and data privacy leadership team' as a specific team composition, but this is a functional team name, not a composition listing distinct departments like the IS-list example 'Legal, Compliance, and Finance.' No genuine IS-list facts remain after filtering.


reasoning: The paragraph is classic IT-dependence boilerplate that the disambiguation rules explicitly categorize as None/Other: it states reliance on IT systems, lists hypothetical impacts of cybersecurity incidents ('could disrupt,' 'could result in'), and includes a cross-reference to Risk Factors. There is no substantive disclosure of governance, management roles, internal processes, or actual strategy integration—only generic forward-looking language about what might happen. Annotators A and C correctly applied the codebook; Annotator B erred by treating hypothetical impact language as substantive Strategy Integration content when the disambiguation rules specify that paragraphs merely establishing IT reliance without describing programs, processes, or strategy fall under None/Other.


specificity_confidence: medium

reasoning: The paragraph describes the company's overall risk management structure—how it flows from the company through its Manager to an IT Service Provider—with vendor relationships as a component of that process rather than vendor oversight being the central topic. Annotators A and C correctly identified the category but missed that 'no employees' is a specific headcount (zero) on the IS-list, making this Firm-Specific rather than merely Sector-Adapted. Annotator B correctly identified the firm-specific fact but misapplied the category by treating vendor reliance as the central topic when the paragraph is primarily about the company's risk management structure.


reasoning: Annotator C correctly applied the codebook. The paragraph contains no IS-list facts: (1) 'third-party security expert consultant' is an unnamed entity on the NOT list; (2) 'over 30 years of experience' is an approximate duration about an unnamed consultant, not an exact duration or hard verifiable number; (3) 'regular updates' is a generic cadence; (4) 'advanced security tools' are unnamed. Annotators A and B erroneously counted 'over 30 years' as a specific fact, but the IS list requires 'exact durations' and the NOT list excludes unnamed entities. After filtering, zero IS-list facts remain, so Generic Boilerplate is correct.


reasoning: The paragraph's primary content describes the company's internal cybersecurity risk management processes—ERM integration, IT security team reviews, penetration testing, and vulnerability ranking—with board oversight mentioned as one component. Under the disambiguation rules, 'We conduct quarterly risk assessments; the board is informed' maps to RMP when process is primary content. Annotators B and C correctly applied this distinction; Annotator A over-weighted the Audit Committee mentions while underweighting that the bulk of the paragraph (especially examples i-iii) describes operational risk management processes. CTO is a valid IS-list fact; Audit Committee is explicitly on the NOT list.


reasoning: Annotator A correctly applied the disambiguation rules. The paragraph's first three sentences describe internal processes (application design, backup systems, monitoring, anti-malware, training, audits, response teams), while the Head of IT mention appears only in the final two sentences as attribution. Applying the test: removing the role mention, the paragraph still makes complete sense as a process description. Annotators B and C overemphasized the brief role mention at the end, treating it as the primary content when it is incidental to the dominant process description.


reasoning: The grammatical subject is clearly 'Senior management,' which is responsible for creating and recommending risk appetite metrics. The Board is merely mentioned as the recipient of management's recommendations—not as the active subject taking oversight actions. Annotators A and C correctly applied the codebook's grammatical subject test, while Annotator B erroneously classified this as Board Governance despite the Board not being the subject performing any action. No firm-specific facts exist—'Senior management' and 'Board of Directors' are both generic references on the NOT-list.


reasoning: Annotator B correctly applied both the category disambiguation rules and specificity criteria. The CEO is mentioned briefly at the start, but the dominant communicative purpose is describing the incident response team structure—an internal process. Applying the disambiguation test: removing the CEO mention leaves a coherent process description about the cross-functional incident response team, confirming RMP. Annotators A and C erred on category by treating the brief CEO attribution as the primary content. On specificity, Annotator A incorrectly concluded the paragraph was generic boilerplate by missing that the specific team composition (cybersecurity, IT support, mining operations, software engineering, compliance, legal, contractors, and partners) is explicitly listed as an IS-list fact per the example 'Legal, Compliance, and Finance.' CEO is correctly identified as generic (NOT list), but the team composition is firm-specific organizational detail.


reasoning: Annotators A and B correctly identified Harley-Davidson as a named third-party firm, which qualifies as a specific fact under the IS-list ('Named third-party firms: Mandiant, Deloitte, CrowdStrike'—examples, not an exhaustive list). The Master Services Agreement with a named external entity distinguishes this disclosure from generic boilerplate. Annotator C erred by dismissing the Harley-Davidson mention and focusing solely on the NIST framework, which alone would yield Sector-Adapted, but the named third-party firm elevates it to Firm-Specific. Only one hard verifiable fact exists (Harley-Davidson), so Quantified-Verifiable is not reached.


reasoning: The paragraph describes how Liberty Broadband manages its cybersecurity risks—by participating in Liberty's processes for assessing, identifying, and managing threats. This is clearly about the company's own internal risk management processes, which happen to be provided through a services agreement with a parent entity. Annotators A and C correctly applied the codebook: the disambiguation rules specify that third parties supporting an internal process (like consultants for penetration testing) fall under Risk Management Process, not Third-Party Risk. Third-Party Risk requires vendor oversight to be the CENTRAL topic (e.g., due diligence programs monitoring vendor controls), which is absent here. Annotator B incorrectly characterized the parent entity relationship as 'vendor oversight.' For specificity, no facts from the ✓ IS list survive filtering—Liberty and GCI are related entities/company self-references (✗ NOT list), and no dates, named external firms, tools, or verifiable specifics appear.


reasoning: Annotators A and B correctly applied the codebook. The paragraph contains no IS-list facts: 'Layer 7 firewall solutions' is a technical category (OSI layer description), not a named product like 'Splunk' or 'CrowdStrike Falcon'; 'antivirus/antimalware/patch system' and 'internal cloud system' are generic system descriptions, not named tools; 'monthly' is a generic cadence without exact dates; and no cybersecurity-specific roles (CISO, CTO, CIO), named frameworks, certifications, or quantified facts appear. Annotator C erroneously treated generic technology descriptions as specific facts, but the IS list explicitly requires named products/tools, not technology categories.


reasoning: Annotators B and C correctly applied the codebook. After filtering against the NOT list, no IS-list facts remain: 'Information Systems Security department' is a generic department name (not a cybersecurity-specific role like CISO, CTO, or VP of IT/Security); 'third-party security vendors' are unnamed entities; 'quarterly' is a generic cadence without exact dates; 'penetration testing' and 'simulated phishing tests' are common practices; 'generative artificial intelligence tools' is a tool category, not a named product; and 'ProAssurance' is a company self-reference. Annotator A erred by counting the department name as a specific fact, but the IS list explicitly enumerates individual cybersecurity roles (CISO, CTO, CIO, VP of IT/Security, Information Security Officer, Director of IT Security), not generic department names that could appear in any company's disclosure.


reasoning: This paragraph is forward-looking risk factor language describing hypothetical consequences of potential cyberattacks using conditional phrasing ('could result in,' 'may also result,' 'could have'). It does not discuss actual strategy, insurance coverage, budget allocation, or a real materiality assessment—it is generic 'what could go wrong' boilerplate. The disambiguation rules explicitly state that forward-looking boilerplate belongs in None/Other, and paragraphs that only establish potential risks without describing programs, processes, or strategies should be classified as None/Other. Annotator C correctly identified this as lacking substantive disclosure; Annotators A and B erroneously treated hypothetical risk language as 'discussion of business/financial impact' when the codebook requires actual discussion of strategy elements, not speculative consequences.


reasoning: Annotators A and C reached the correct conclusion. The paragraph contains three QV-eligible facts: 'nearly 30 years of experience' (specific number), 'more than 20 years of experience' (specific number), and 'CISSP certification' (named certification). Per the disambiguation rules, years of experience and named certifications both count toward the 2-fact QV threshold. Annotator B incorrectly stopped at Firm-Specific, failing to count the multiple hard verifiable facts present.


reasoning: Annotators B and C correctly applied the disambiguation rules, which explicitly state that 'generic IT-dependence language' like 'our IT systems are important to operations' should be classified as None/Other. This paragraph merely states that Luxfer depends on IT systems and a cyber-attack 'could' cause disruption—this is forward-looking risk factor boilerplate, not an actual discussion of business/financial impact strategy, insurance, budget, or materiality assessment. Annotator A erred by conflating the mention of potential operational impacts with substantive Strategy Integration disclosure; the codebook requires actual discussion of these topics, not hypothetical risk factor language that could appear in any company's filing.


reasoning: The paragraph is a malformed cross-reference to Item 1A risk factor disclosures with no substantive cybersecurity disclosure. The materiality language ('materially affected or are reasonably likely to materially affect us') is embedded in what appears to be regulatory/instructional text describing disclosure requirements, not an actual materiality assessment or conclusion by the company. The only substantive statement is generic boilerplate that something 'is an important part of our risk management processes and an area of focus for our Board and management.' I agree with Annotators A and C; Annotator B incorrectly treated regulatory language as an actual materiality disclosure.


reasoning: The paragraph describes the company's internal cybersecurity posture—including lack of personnel, processes, and resources—and the incident response reporting mechanism, which fits Risk Management Process. For specificity, after filtering NOT-list items ('Management', 'Board of Directors', 'incident response plan' as generic program name, 'Initial Public Offering' as non-specific temporal reference), zero IS-list facts remain. Annotators A and C correctly identified this as Generic Boilerplate; Annotator B erred by not filtering generic program names from the IS-list.


reasoning: The paragraph correctly classified as Risk Management Process for ERM integration. For specificity, after filtering NOT-list items, zero IS-list facts remain: 'enterprise risk management committee' is a generic program name without unique identifier (comparable to 'incident response plan' on the NOT list), 'Board of Directors' and 'management' are generic governance terms, 'annually' and 'regularly' are generic cadences without exact dates, and 'executive leadership' is not a cybersecurity-specific role. Annotators A and C correctly applied the codebook; Annotator B erroneously counted the ERM committee description as a specific fact when it lacks the distinguishing characteristics (proper noun capitalization, unique identifier like 'CIRP') that would elevate it from generic to firm-specific.


reasoning: I agree with Annotator B. Applying the disambiguation test: if the role mentions are removed, the paragraph still functions as a complete process description covering program implementation, multidisciplinary team deployment, monitoring, detection, mitigation, remediation, and incident reporting. The IT Engineer and Cybersecurity Council are incidental attributions—the vehicles through which processes are described—not the primary focus. The paragraph describes what the company DOES (internal processes), not the identity, qualifications, or reporting structure of managers. Annotators A and C incorrectly treated the grammatical subjects as determinative when the codebook requires examining the DOMINANT communicative purpose. The paragraph contains three IS-list facts (IT Engineer, Cybersecurity Council, Risk Management Committee) but no QV-eligible facts (no dates, amounts, headcounts, named firms, or named tools), confirming Firm-Specific specificity.


reasoning: The paragraph describes the company's internal Information Security Program controls and processes, clearly fitting Risk Management Process. For specificity, the team composition 'information security, technology, IT and operations personnel' qualifies as a firm-specific fact per the IS list example ('Legal, Compliance, and Finance'). Annotator A correctly identified this; Annotator B incorrectly concluded no facts exist; Annotator C reached the right label but relied on 'daily' scans, which falls under generic cadences on the NOT list. No QV-eligible facts (no named firms, products, dates, or quantities) exist, so Firm-Specific is the ceiling.


reasoning: Annotators A and B correctly applied the codebook. After strict filtering, no IS-list facts remain: 'Board' and 'Audit Committee' are generic governance (NOT list); 'Advisor', 'Sub-Advisor', 'Dealer Manager', and 'Inland' are company/entity self-references (NOT list); and 'incident response plan' is explicitly listed as a generic program name without a unique identifier. Annotator C provided no justification for Firm-Specific and appears to have counted NOT-list items as specific facts, which is the #1 annotator error identified in the disambiguation rules.


content_category: Strategy Integration

reasoning: The paragraph's dominant communicative purpose is delivering a standalone materiality conclusion (no prior incidents have materially affected the company) and discussing potential future business/financial impact, which aligns with Strategy Integration. Annotators B and C correctly applied the codebook: the paragraph addresses materiality and business implications rather than describing HOW risks are operationally assessed or mitigated. Annotator A erred by focusing on the incidental mention of 'cybersecurity risk management program' without recognizing that the paragraph does not describe operational processes—it merely acknowledges the program's existence in service of a materiality disclaimer. The specificity is correctly identified as Generic Boilerplate by all annotators, as the text contains no IS-list facts—no named roles, frameworks, dates, products, firms, or quantified details.


reasoning: Annotator B correctly applied the disambiguation rules: the VP is mentioned once at the beginning, followed by extensive process description (monitoring, tools, scans, audits, vulnerability assessments, tabletop exercises). The role mention is incidental attribution. Applying the codebook test—remove the role mention and the paragraph still makes complete sense as a process description—confirms Risk Management Process. Annotators A and C focused on the opening sentence without applying the disambiguation test, incorrectly treating brief role attribution as the primary content. For specificity, VP Cybersecurity, CIO, and CTO are all cybersecurity-specific titles on the IS-list, but no hard verifiable facts (dates, dollar amounts, headcounts, named third parties, named tools) reach the Quantified-Verifiable threshold.


reasoning: The paragraph names MLPS (Multi-Level Protection Scheme), which is China's recognized cybersecurity framework—a named standard that qualifies for Sector-Adapted. However, no facts from the IS list are present: IT Manager is not a cybersecurity-specific title as explicitly defined in the codebook (which lists CISO, CTO, CIO, VP of IT/Security, Information Security Officer, Director of IT Security), CFO is on the NOT list as generic C-suite, and JiuGe Technology is a company self-reference. Annotator B correctly applied the codebook; Annotators A and C incorrectly counted IT Manager as a firm-specific fact when it does not appear on the IS list of cybersecurity-specific titles.


reasoning: The disambiguation rule directly applies: the VP of IT is mentioned once at the beginning in a relative clause, followed by three sentences describing the Cybersecurity Program's processes (monitoring, prevention, detection, mitigation, use of experts, threat intelligence sources, security tools, executive briefings). Removing the VP mention leaves a coherent process description, confirming the role mention is incidental. Annotator B correctly applied this test; Annotators A and C treated the opening attribution as the primary focus when it's actually a brief lead-in to extensive process content.


reasoning: The paragraph describes the company's internal evaluation process for third-party service providers, with 'We' as the grammatical subject performing the evaluation action. This fits Risk Management Process as a risk assessment activity. The codebook tiebreaker states 'Internal processes mentioning vendors as one component → Risk Management Process.' While Annotator C correctly notes the first sentence focuses on third-party providers, the disambiguation rules distinguish between third parties supporting an internal process (RMP) versus a dedicated vendor oversight program being the topic (TPR). This paragraph describes evaluation as part of the company's risk management rather than a standalone vendor oversight program. However, the case is borderline because the first sentence is entirely vendor-focused, warranting MEDIUM confidence. All annotators correctly identified Generic Boilerplate specificity—no IS-list facts exist (Board of Directors, Audit Committee, and 'third-party service provider' are all generic terms on the NOT list).


reasoning: Annotator C is correct. The disambiguation rules explicitly state to ask 'who is the grammatical subject?' The CEO is the grammatical subject in all three sentences ('CEO plays a pivotal role,' 'They provide comprehensive briefings,' 'These briefings encompass'). The Audit Committee is the object receiving the briefings, not the subject taking oversight actions. Annotators A and B incorrectly prioritized the Audit Committee's presence over the grammatical subject test. For specificity, all annotators correctly identified this as Generic Boilerplate: CEO and Audit Committee are both on the NOT list as generic roles, and 'once per year' is a generic cadence without a specific date.


reasoning: The disambiguation rule explicitly states that 'Board or board committee taking oversight actions (receiving briefings, reviewing risks) → Board Governance.' Here, the Audit Committee is receiving briefings/updates on cybersecurity incidents, which constitutes an oversight action. While 'Management' is the grammatical subject performing the action, the paragraph's focus is on the board committee's oversight function rather than on management qualifications or reporting lines. Annotators A and C correctly identified this as Board Governance; Annotator B incorrectly prioritized the grammatical subject over the explicit disambiguation rule about receiving briefings. All annotators correctly identified Generic Boilerplate specificity, as 'Management,' 'Audit Committee,' and 'executives' are all generic terms on the NOT-list, with no dates, named roles, frameworks, or verifiable facts present.


reasoning: The paragraph describes the company's approach to cybersecurity risk management being integrated into a broader framework, which is a Risk Management Process topic. The Board is mentioned only as overseeing the process, not as the grammatical subject—the subject is 'Our approach to cybersecurity risk management.' Annotators A and C correctly applied the codebook; Annotator B incorrectly identified the Board as the grammatical subject when it merely receives a passive mention at the end. No IS-list facts exist—'Board of Directors' is generic governance on the NOT list, and no frameworks, roles, dates, or quantified details appear.


reasoning: Annotator B is correct. The disambiguation rule asks whether the paragraph describes actual cybersecurity activities (monitoring, assessment, vulnerability management, training programs). This paragraph only states that the company collects/stores data, that IT systems are important, and that security is critical—without describing any actual processes or activities the company undertakes to manage cyber risk. Annotators A and C incorrectly interpreted the mention of 'secure maintenance' and CIA triad concepts as describing risk management processes, but the paragraph contains no substantive disclosure of what the company DOES about cybersecurity—only contextual statements about data and IT systems that could apply to virtually any company.


reasoning: The paragraph names the NIST Cybersecurity Framework (NIST CSF), which qualifies for Sector-Adapted specificity. However, no genuine IS-list facts are present: 'Centuri' is a company self-reference (not a specific fact per the codebook), 'quarterly' is a generic cadence without exact dates, 'Executive Leadership Team' is generic C-suite rather than a cybersecurity-specific title like CISO, 'Cybersecurity Risk Register' is a generic program name without a unique identifier (unlike 'CIRP' in the examples), and 'third parties' are unnamed entities. Annotators A and C correctly applied Sector-Adapted; Annotator B incorrectly elevated to Firm-Specific by counting items that appear on the NOT-list.


reasoning: The paragraph describes internal cybersecurity risk management processes, correctly classified as Risk Management Process. After filtering, zero IS-list facts remain: 'cybersecurity risk management program' and 'enterprise risk management program' are generic program names without unique identifiers; 'internal team of cybersecurity professionals and data privacy professionals' is a generic description of professional types, not a specific team composition like 'Legal, Compliance, and Finance'; 'external experts' are unnamed entities. Annotators A and B correctly applied the codebook; Annotator C erroneously counted the generic team description as a specific fact.


reasoning: The paragraph names the NIST Cybersecurity Framework, qualifying for Sector-Adapted, but contains no genuine IS-list facts. The phrase 'product development and technology management teams' describes which departments cybersecurity integrates with—not a specific team composition like 'Legal, Compliance, and Finance' that would identify a unique organizational structure. The 'in-house cybersecurity experts' and 'third-party providers' are unnamed entities on the NOT list. Annotators A and B correctly applied the codebook; Annotator C claimed 'specific team integrations' as firm-specific but this does not meet the IS-list criteria for team composition.