Net-Centric Data Strategy, Department of Defense Net-Centric Data Strategy, Department of Defense DoDNCDS _7f81f66c-9a35-4b6c-ac01-c993d3ecb9be All data is advertised and available for users and applications when and where they need it. In this environment, users and applications search for and “pull” data as needed. Alternatively, users receive alerts when data to which they have subscribed is updated or changed (i.e., publish subscribe). Authorized users and applications have immediate access to data posted to the network without processing, exploitation, and dissemination delays. Users and applications “tag” data assets with metadata, or data about data, to enable discovery of data. Users and applications post all data assets to “shared” space for use by the Enterprise. _6bf28809-eb13-4a88-89dc-85b4b40a4a3e This document describes the Net-Centric Data Strategy for the Department of Defense (DoD), including DoD intelligence agencies and functions. It describes a vision for a net-centric environment and the data goals for achieving that vision. It defines approaches and actions that DoD personnel will have to take as users—whether in a role as consumers and producers of data or as system and application developers. This Strategy will be followed by a subsequent directive and guidance on implementation details. _6f1cf763-f436-4e7f-b2dd-39cef7a4cce1 MAKE DATA VISIBLE Advertise and make all data assets "visible” by providing metadata, which describes the asset. _0f2977b1-36d3-4ea9-b536-e7e3de5d0ad1 3.1 Users and applications can discover the existence of data assets through catalogs, registries, and other search services. All data assets (intelligence, nonintelligence, raw, and processed) are advertised or “made visible” by providing metadata, which describes the asset. Data Sharing Post Data to Shared Spaces _32641709-c2f4-40f6-ac1b-15d686b62cf5 3.1.1 Users and applications will migrate from maintaining private data (e.g., data kept within systemspecific storage) to making data available in community- and Enterprise-shared spaces (e.g., servers and services available on the Internet). These shared spaces will act as repositories where users and applications can submit, or post, data assets to the enterprise. The shared spaces will provide storage and serving mechanisms. Enterprise-shared spaces will be maintained, secured, and staged as necessary to support the Department’s missions. Data that is posted to shared spaces will be advertised via the associated metadata and will be discoverable with enterprise search tools. ef74f650-5d80-4184-891a-830cee6d45cc 37b38c9a-ea6c-44ee-aa5a-249263645e48 Discovery Metadata Associate Discovery Metadata With Data Assets _764c43a1-ea9a-43c2-b6ee-d574c3193bae 3.1.2 To facilitate discovery of data assets, users and applications will provide discovery metadata, in accordance with the DoD Discovery Metadata Standard (DDMS), for all data posted to shared spaces. The DDMS will provide a common set of structured attributes that support discovery of data assets using search tools. COIs and asset producers determine the desired level of discovery for a data asset, e.g., discovery of a database or a record within a database, discovery of a document or a paragraph within a document. The initial focus of the DDMS is to aid in the discovery of data assets as a whole; hence, the discovery metadata in the DDMS will not always be required for individual records or elements. For exa mple, the discovery metadata will always indicate the existence of a database containing certain kinds of information but may or may not identify the contents of specific database elements. The DDMS does not preclude the use of other metadata processes or standards. For example, record- level database tagging and in-line document tagging are common practice today to support various Department objectives. These tagging initiatives will only have to enhance their existing processes to include the DDMS for Enterprise discovery. The DDMS will be adopted and implemented across the DoD components for new and existing data assets. Figure 7 illustrates the logical layers and elements of the DDMS. This standard will be registered in the DoD Metadata Registry. Enterprise visibility of a data asset is promoted when its discovery metadata complies with the DDMS. The core layer of the DDMS represents those attributes of a data asset that can be commonly described across the Department, regardless of the type of data asset or community to which it is applied. It is important to note that not all elements in the core layer are mandatory. The set of mandatory elements will be small yet robust enough to generate high value-added metadata to support Enterprise discovery. The extensible layer provides a mechanism for COIs to extend the core layer of the DDMS to support mission-specific metadata requirements. d01a8778-3133-4325-97b9-a543bdbd7dfa 84986c5e-9978-487a-ab62-385174e1a28f Metadata Catalogs Create and Maintain Catalogs _f0810a9a-f7c7-4099-b6c7-870e5bdf03da 3.1.3 Metadata catalogs will advertise the existence of shared data and will contain information about all data assets contained in the associated shared space (including databases, system output files, web pages, documents, and access services). At a minimum, the mandatory discovery metadata elements in the DDMS must be represented within metadata catalogs for any data asset posted to a shared space. COIs will establish and maintain catalogs. Each catalog may be organized according to the community-defined ontology. An Enterprise catalog will be established that links to community catalogs, effectively creating a “catalog of catalogs.” The Enterprise catalog will also contain metadata for data posted by users and applications without a specific COI affiliation. Catalogs will be searchable by applications or through user-friendly, web-based interfaces. The web-based interfaces will have a consistent look and feel and will support posting of metadata to the catalog and data to the shared space. The catalogs will also be searchable, either manually or automatically via agents, through application programming interfaces. All metadata catalogs will adhere to Enterprise discovery interface standards to allow searches within a catalog or across catalogs. 38f325b3-d07b-42d0-afbb-b779b564b457 40edaaa0-ba88-4ec6-ab48-44f6370fe04f Metadata Registry Register Metadata Related to Structure and Definition _fa2ae4b9-6284-45fb-a89f-7a42e0987344 3.1.4 The DoD Metadata Registry will contain all metadata related to data structures, models, dictionaries, and schemas. The purpose of the Registry is to give developers and architects visibility into methods to compose and encode data and to share usage across the Department. Registration of such metadata is critical to achieve the data goals of interoperability and understanding by promoting semantic and structural understanding. 887f7ed3-6818-4f7d-90a1-59d18f308888 af90a032-4761-43ae-9b20-515d8f8cbb06 Inventory Inventory Data Assets _de2968d9-cce7-44d9-b4c4-ca705e0f5d47 3.1.5 During transition to the new DoD Net-Centric Data Strategy, COIs may identify and prioritize key data assets and services within their domain. These data assets or services may already exist or they may be in development. In either case, COIs will identify the data assets and services that must be made compliant with the data approaches. This prioritized list of data assets and services will provide a focus for near-term COI initiatives to create metadata to advertise the data and ensure that the data is available in shared space. 6a956362-a063-4abc-ac17-3b27fda60a6b dcc0699c-0e84-45f8-8c24-224ab17fdbcd Data Accessibility MAKE DATA ACCESSIBLE _bbc70a1f-3658-4223-a8e7-58d4e278e8f0 3.2 Users and applications post data to a “shared space.” Posting data implies that (1) descriptive information about the asset (metadata) has been provided to a catalog that is visible to the Enterprise and (2) the data is stored such that users and applications in the Enterprise can access it. Data assets are made available to any user or application except when limited by policy, regulation, or security. Data Access and Sharing Create Shared Spaces and Data Access Services _8d3b0892-08f8-4f5d-b08f-969716c9068f 3.2.1 Shared spaces—virtual and actual—will be created to provide a “store and serve” mechanism for data assets. In addition, data access services will be created to help facilitate access to database stores, business logic processes, and system data. Data access services are any mechanisms that help expose data that is not otherwise available to users and applications. For example, a data access service may be a registered, accessible software interface that allows users and applications to extract information from an inventory database. b09e24e3-6c7b-40de-80ca-24696622dc86 d92d63c6-80fb-4391-ac74-15663abb8efd Security-Related Metadata Associate Security-Related Metadata _9bc7406d-5ead-4609-8daf-d91280dcb513 3.2.2 Security-related metadata will be provided for each data asset as defined by the Security Descriptors element set within the core layer of the DDMS (see Figure 7). Systems will be able to control access to assets based on classification metadata. GES will allow data that was typically inaccessible as a result of implementation barriers (e.g., distinct networks based on data classification and prearranged accounts for access) to be available to users and applications that have appropriate access needs. These services will permit access using security metadata, technologies such as public key infrastructure (PKI), and role- and permission-based access processes if adopted. 40b5a7eb-f600-4bfd-9948-22e956242e86 82ae153d-de18-4871-8f31-5fe199455e2a Data Management INSTITUTIONALIZE DATA MANAGEMENT _093be162-c9da-4b02-8083-66791aa7abaa 3.3 Data approaches are incorporated into Department processes and practices. The benefits of Enterprise and community data are recognized throughout the Department. Leadership and Governance Govern Data Processes With Sustained Leadership _795b1b56-c2d8-42c0-a4b1-10126651352a 3.3.1 Best-practice studies have shown that cultural change is most effectively achieved when there is sustained leadership commitment and institutionalization of new processes. The DoD CIO will establish a DoD-wide net-centric governance process to promote and sustain successful data management practices across COIs by reviewing and sharing metrics, best practices, and incentive structures. This DoD-wide net-centric governance process will also provide oversight of net-centric infrastructure development efforts. 966afacf-b61d-4d2a-9fee-1f8e60c4c489 eb40436a-ed00-4f5d-9038-27754d8312e7 Processes and Practices Incorporate Data Approaches Into Department Processes and Practices _95be375e-259f-425d-ba4d-ab182febb6fa 3.3.2 DoD Components will recognize and fund implementation of data approaches (e.g., providing metadata, defining and registering metadata structures, and posting data). Planning, programming, and budgeting system processes and policies, as well as acquisition processes and policies, will be updated to reflect these approaches. 85c48516-7f7d-4e56-b42c-ebece622d825 42790295-a66a-431c-ae31-a61c2b8455da Advocacy, Training, and Education Advocate, Train, and Educate in Data Practices _db80f34b-c834-4446-abc8-cd00de3a04c3 3.3.3 Best practices show that new operating practices are assimilated more quickly when consistently promoted. The DoD CIO will continue to conduct the Data Strategy awareness campaign across the Department, promoting and educating all users about their responsibilities and the benefits of participation. The necessary tools to implement these approaches (e.g., DoD Metadata Registry, catalogs, and shared spaces) will be identified and implemented. As these tools become available, training and manuals will be provided. 5cd0d226-8783-4d54-8a71-5d34f8d5d2e5 8f6d56b1-66c2-443a-a049-86c18036b34c Metrics and Incentives Adopt Metrics and Incentives _7b110602-7544-45bd-bfad-735cf63503dc 3.3.4 The DoD-wide net-centric governance process will identify incentives and metrics. Users and system developers will be guided by incentives to encourage and foster participation. Incentives are intended to prompt users and developers to contribute to increased data visibility across the Enterprise. Incentives may include rapid or continued funding for initiatives that implement these data approaches (e.g., providing metadata in accordance with the DDMS). Incentives may also include Enterprise-visible “top products” lists that cite specific sources that have been especially valuable to users. Metrics will be collected to track implementation and application of the approaches. Metrics will be helpful in evaluating usage to ensure participation across the Department. Metrics also serve as a means to evaluate the effectiveness of the overall Data Strategy. Measurement techniques will be developed to ensure that metrics are captured in a useful and consistent manner. Moreover, multiple publication methods such as metrics web sites and Enterprise- and COI-specific metrics reports will be made available Department-wide to promote awareness of data management successes and areas requiring improvement. 1926a3d1-f0eb-4edf-b264-16b802c2b1a7 1aac2761-afe7-47d3-9e68-8d565016f425 Understandable Data ENABLE DATA TO BE UNDERSTANDABLE _41439c5a-50d6-4b55-8e73-7adf4412cd17 3.4 Users and applications can comprehend the data, both structurally and semantically, and readily determine how the data may be used for their specific needs. COI-Specific Ontologies Define COI-Specific Ontologies _2e9dfa2f-0065-4910-8884-52affb65fb3d 3.4.1 COIs will be encouraged through metrics and incentives to develop an ontology that best reflects the community understanding of their shared data. Ontologies include data categorization schemes, thesauruses, vocabularies, key word lists, and taxonomies. Ontologies promote semantic and syntactic understanding of data. For example, taxonomies enhance discovery by providing a hierarchical means of searching for data while providing users and applications with additional insights about data assets by indicating their placement among other data assets. COIdeveloped vocabularies will define terms used in describing data assets, and the thesauruses will identify related terms to assist translation services. Any community that establishes an ontology will publish it to the DoD Metadata Registry to increase understanding across the Enterprise and promote possible reuse. c1e1cb8f-3cf3-4deb-9d94-1849eeb74f75 d49a2a8e-30c0-4fb4-b29e-407a54f8046c Content-Related Metadata Associate Content-Related Metadata _8e97a350-1f00-4fa5-8a9e-cfde1c585ecb 3.4.2 The summary content descriptors element set of the DDMS (see Figure 7) is specifically aimed at providing “content-related” details about data assets. Content metadata provides topics, keywords, context, and other content-related information. Content metadata gives users and applications insight into the meaning and context of the data. Content metadata provides a basis for search engines to perform searches for data assets that address specific topics. f6d86588-6eee-43d3-80b1-8374e466c646 93825a89-9652-4e5b-84c5-a6a2a7a467c6 Format-Related Metadata Associate Format-Related Metadata _b5330b1e-e2a5-49c2-ae1c-8dca72abf621 3.4.3 The format descriptors element set of the DDMS (see Figure 7) is used to describe details pertaining to the format of the associated data asset. The format descriptors are useful when trying to understand the physical manifestation of an asset. For example, the format descriptors will provide information regarding the type of digital file (e.g., a Joint Photographic Experts Group [JPEG] image, or an Audio Interchange File Format [AIFF]). In addition, the format descriptors contain optional information that describes the extent of the asset, such as file size, bit rate, and dimensions. Format-related metadata allows users and applications to narrow down information searches and to select products that meet their particular operating constraints (e.g., a user who is able to view only Graphic Interchange Format [GIF] images would not want to pull a JPEG image). e78468fc-5ded-4d95-86fc-53f23ade0d8e cc07a635-1d72-43d4-a176-8fe65945b177 COI-Specific Metadata Define COI-Specific Content-Related Metadata _15e5c674-2e8a-4298-9915-44a35c8f6af4 3.4.4 To improve understanding, an extension of the discovery metadata standard is reserved for domain-specific, or COI-specific, metadata. This is represented as the extensible layer of the DDMS in Figure 7. With this extension layer, COIs will be able to provide context relevant to their particular domain area and still be able to participate in Enterprise-wide search and discovery. COIs will be required to register their COI-specific content metadata requirements in the DoD Metadata Registry. These COI-specific metadata requirements may then be integrated into appropriate Enterprise and community services such as search and mediation. 41527a1f-2d39-454d-8076-2374667974a1 86bb542c-7928-44fd-87ac-09c6b0595b20 Trusted Data ENABLE DATA TO BE TRUSTED _eb7754f4-ed5a-468a-a36e-607344f2e4af 3.5 Users and applications can determine and assess the authority of the source because the pedigree, security level, and access control level of each data asset is known and available. Pedigree and Security Metadata Associate Data Pedigree and Security Metadata _8a34fe6d-8265-413e-b610-f3dfcc1827e2 3.5.1 The Resource Descriptors elements of the DDMS (see Figure 7) allow identification of the author, publisher, and sources contributing to the data, allowing users and applications to assess the derivation of the data (i.e., data pedigree). This metadata allows users and applications to select data from known sources. Reliable and quality sources will become more widely used, enhancing overall data quality throughout the Enterprise as more data sources become visible. The Security Descriptors elements of the DDMS (see Figure 7) allow security and privacy markings consistent with established standards where applicable. For information assurance (IA) and security, GES will provide auditing tools that can track access, by individual user, of each data asset. GES may also provide access control to data assets based on security markings in the metadata. 84cc6be1-66af-4563-b5be-4860b75073e3 1cdc7657-98d7-47be-8c28-8a0c37ee2190 Authoritative Sources Identify Authoritative Sources _9de55d5f-0426-486e-aa1a-d1ce1794baa2 3.5.2 COIs may identify authoritative sources for key data assets in their domain. The community will publicize their identified authoritative sources to the Enterprise, thus allowing users and applications to evaluate and understand the community- implied authority of data sources. COIs may have to resolve potentially conflicting sources and, where appropriate, coordinate with the DoD-wide governance body to identify authoritative source(s). e12ae584-8458-4e3a-8fec-2fac93c747df 1a1c8e37-a17d-44fb-8b8c-52885e1864f2 Data Interoperability SUPPORT DATA INTEROPERABILITY _8740c845-30e4-4020-b270-d875bd542c2a 3.6 Many-to-many exchanges of data occur between systems through interfaces that are sometimes predefined or sometimes unanticipated. Metadata is available to allow mediation or translation of data between interfaces, as needed. Metadata Registration Register Metadata _1e28c040-ddaf-4424-a4d0-ff447dbf3a2c 3.6.1 Registration of metadata (e.g., eXtensible Markup Language [XML] components, database segments, and data dictionary elements) is an important activity to support interoperability in a net-centric environment. COIs will register their metadata components in the DoD Metadata Registry. Registering metadata components to the DoD Metadata Registry supports many-to-many interoperability by providing system architects and developers with insight into existing data schemas that they can employ and extend. 809f6392-316c-45b0-8269-dd8a766bc8e1 af94a6f2-5bdb-4a1f-94d6-3adf41886298 Format-Related Metadata Associate Format-Related Metadata _6226d56e-c4f1-44d0-afdb-8694e7031e22 3.6.2 Users and systems can employ the elements of format descriptors to specify the extent (e.g., size and dimension), type, and physical manifestation of assets. The format descriptors element set of the DDMS (see Figure 7) supports interoperability by allowing systems and users to determine the physical manifestation of data assets, which in turn helps to identify which tools and capabilities are required to use the asset. 35a8cc22-0fb0-401f-965d-2d929b9dd9e6 3336d28f-8fdf-4fd9-bc25-f2eae025838e System Interfaces Identify Key Interfaces Between Systems _1390f5cf-339a-4391-b0e2-711910c16d28 3.6.3 Interface engineering in an environment of many-to-many exchanges requires an unrealistic degree of interface control and an enormous commitment of resources. To facilitate interoperability within a community, COIs can determine the appropriate focus and level of data standardization within their community. This decentralized, distributed approach to interoperability ensures that key interfaces and data structures are controlled when tightly engineered interfaces are required. COIs will register metadata that results from interoperability activities in the DoD Metadata Registry. Mediation services will use the registered metadata to facilitate system interoperability between unanticipated interfaces as needed. 235eb770-6312-41f5-bf16-4a58b6482f36 839051ed-7445-4734-b698-1bb9996b4fd2 Interface Standards Comply With Net-Centric Interface Standards _bee581e7-2f1d-425a-a22d-da7ad9eeeed4 3.6.4 Developers will be responsible for adhering to published net-centric interoperability standards, including data standards where applicable. Successful discovery and interoperability of data assets depend on compliance with metadata standards (i.e., DDMS) and data exposure standards (e.g., GES discovery interface standards). For example, data assets that are maintained by the Department’s Records Management functions must provide a means for the Enterprise discovery capability to query the inventory of their stored records. In doing so, these records management applications should employ the DDMS to respond to Enterprise discovery queries. 0d427e39-e5da-411e-b8e9-c7a1fd560937 33b804b0-8221-4879-bffe-d44bd32bb1f0 User Needs BE RESPONSIVE TO USER NEEDS _a3ae4f46-a6b8-4be2-89a7-7d879245c7e3 3.7 Perspectives of users, whether data consumers or data producers, are incorporated into data approaches via continual feedback to ensure satisfaction. User Involvement Involve Users in COIs _d688fe91-1438-4b98-b5e3-ec7ed68bec22 3.7.1 As described in Section 2.1, institutional COIs in particular are focused on ensuring the implementation of these approaches. Institutional COIs will take the lead in creating catalogs, defining ontologies, and developing COI-specific metadata. To adequately reflect user needs, these COIs must engage a range of known users and developers in these activities. e5b1537c-c285-4bde-bef9-a95745518747 b5b02d27-1a85-46ce-8b56-b87d4fe26109 User Feedback Establish a Process To Enable User Feedback _3be85be2-9115-4cc5-af98-687d47970f90 3.7.2 COIs, under the DoD CIO Enterprise-wide net-centric governance process, will establish processes to evaluate and refine the user experience. Users may provide ratings for data sources, catalogs, or services, and content metadata usability. Ratings may include factors such as ease of use, applicability, or quality. These ratings will be published Enterprise-wide and used to promote participation in posting, identifying, and sharing data assets. Overall, this Departmentwide feedback and ratings process, coupled with improved data asset visibility, will increase the integrity and quality of data. In addition, the feedback process allows COIs and data producers to identify previously unanticipated users and applications. To improve Enterprise data visibility, the process may allow users to identify needed data by publishing a “data want ad” to a community or Enterprise collaboration space. In some cases, the data may be available but not currently visible or accessible. Hence, the source may choose to make it visible or accessible to the user or application. In other situations, providing the data may not be available or cost-effective, and the user’s “want” will remain unfilled. 2f778bc0-d357-47e6-8701-82698ef330c2 9903db6c-88c6-4ef2-8d69-5bbe39084bce 2003-05-09 2010-02-08 http://www.defenselink.mil/cio-nii/docs/Net-Centric-Data-Strategy-2003-05-092.pdf Arthur Colman (www.drybridge.com) colman@drybridge.com Submit error.