Difference between revisions of "DbDIPview"

	Framework for packaging the database Representation Information and pre-configured user-friendly access. Different combinations of Content Data Objects are supported by an automated deployment mechanism. Enables access to the archived databases in the archive reading room for non-technical users.
Homepage:	https://github.com/dbdipview/dbdipview/wiki
License:	EUPL-1.2
Platforms:	linux
Function:	Access,Quality Assurance,Redaction
Content type:	Database

Latest revision as of 13:23, 3 April 2023

Description[edit]

dbDIPview provides simultaneous access to multiple archived databases for non-technical end-users in the electronic reading rooms. To the archivist, it offers a way to work with Representation Information. It deals with:

- representation information that supplements original database data packages,

- automatic deployment of database packages (data objects) and related access packages, including redaction

- access mechanism to the database via a browser menu with report selection and search screen for a selected database that mimics the original application.

Licensing and cost[edit]

Free/open-source EUPL-1.2

Development activity[edit]

dbDIPview has been initially created in 2009 and is continuously being improved to cover requirements by new use cases in the area of long‐term preservation of databases.

Platform and interoperability[edit]

Linux, PHP, Apache2, PostgreSQL

Functional notes[edit]

Before export from the original environment is started, no modification of the data is needed, like de-normalization or the creation of additional views. This is important to ensure data integrity and authenticity.

A database can be stored in SIARD or dbDIPview packaging (based on CSV) format. Complex combinations of data packages are possible.

In the SIP/AIP Audit Phase in the archives, an expert needs to validate the content. For the archivist, the most convenient way is to use a dissemination package. During the validation process, database queries are prepared to show the content of the tables in the same way it is shown in the reports from the original environment. The queries are written in pure (portable) SQL and are stored in long-term storage as part of the Representation Information Package. The queries can be prepared using the information provided by the creator, using some dedicated query builder, or simply manually. When a basic query is ready, the search criteria panel including drop-down fields, and drill down behaviour can be configured. The test serves as proof that the information in the database is well understood, described for future use and is ready for dissemination.

Upon request by an end-user, the order fulfillment process enables that packages are transferred to the dedicated dbDIPview server. There, the information in the database Representation Information Package enables seamless automated deployment of data package(s) of the archived database data objects and setting up of the rendering module. Automated database redaction and creation of additional views are possible at this stage. The user can be redirected via a unique code directly to the menu with queries for the desired database.

When dissemination is not needed anymore, the database can be removed, or it remains available as persistent DIP (DIPp).

Some benefits of this approach:

The quality of ingested data in the SIP is tested during the pre-ingest and ingest process when we prepare the rendering information
After the ingest, no further expert involvement is needed for future use of the content because the tool is archival personnel friendly and allows unattended deployment
The configurable interface enables mimicking the reporting part of the original application
Links to external files are possible from column data
Table content and external files can be added to existing DIPp, thus eliminating the need for redeployment.
Database restoration can be done from SIARD files (via external tools SIARD Suite or DBPTK) or packages with CSV files
Redaction of database tables can be configured as part of representation information to be carried out during deployment
Preconfigured end-user friendly browser user interface
The end-user can download the result of a query as a CSV file
It is possible to combine and link the results, i.e., jump from one report to another one with detailed information
Database column descriptions are shown as infotip
Simple technology minimizes the dependencies and increases maintainability, technical robustness, and supportability for long-term use
PHP code simplifies security audit if it is requested before deployment into the archival environment

Documentation and user support[edit]

Demonstration examples are included with sample databases and related Representation Information.

Usability[edit]

End-users interact with the system through a browser user interface and work with the accessible databases. The hierarchical reports menu enables selection between available reports to access the information without the assistance of an expert. In the results pane, drill-down links to further reports may be available based on column values. BLOB data and external files may be shown. Additional metadata can be shown with each report to provide a detailed description of the given report.

Expertise required[edit]

Basic knowledge of Linux system administration is needed for initial installation and configuration. Familiarity with basic SQL commands and an understanding of database structure is needed to configure the viewer. No special skills are required for the end-user.

Standards compliance[edit]

dbDIPview uses its XML schema for storing information about the queries, and its format for packaging the Representation Information Package and optional CSV package. The queries should not use any SQL dialect to minimize compatibility issues with a target database.

Influence and take-up[edit]

The tool is in production use.

User Experiences[edit]

Some details of database data rendering: https://github.com/dbdipview/dbdipview/wiki/Screenshots

Development Activity[edit]

The tool is stable but new features are added from time to time. All development activity is visible on GitHub.

@@ Line 1: / Line 1: @@
-<!-- Use the structure provided in this template, do not change it! -->
+{{Infobox tool
+|purpose=Framework for packaging the database Representation Information and pre-configured user-friendly access. Different combinations of Content Data Objects are supported by an automated deployment mechanism. Enables access to the archived databases in the archive reading room for non-technical users.
-{{Infobox_tool
-|purpose=A viewer platform for preserved databases
-|image=
 |homepage=https://github.com/dbdipview/dbdipview/wiki
 |license=EUPL-1.2
 |platforms=linux
+|function=Access, Quality Assurance, Redaction
+|content=Database
 }}
-<!-- Note that to use the image field, you should leave the value as {{PAGENAMEE}}.png (or similar) and upload a copy of the image. Hot-linking is not supported. If you don't want an image, just remove that line. -->
+{{Infobox tool details}}
-[[Category:Access]]
-[[Category:Database]]
 == Description ==
-dbDIPview provides parallel access to multiple archived databases for non-technical end-users. It offers a way to configure and store access information based on queries, written in pure (portable) SQL and stored in XML. It deals with
+dbDIPview provides simultaneous access to multiple archived databases for non-technical end-users in the electronic reading rooms. To the archivist, it offers a way to work with Representation Information. It deals with:
-::- packaging representation information
-::- automatic deployment of database packages (data objects) and related access packages
-::- access mechanism to the database via a browser menu with report selection and search screen for a selected database that mimics the original application.
-Before export from the original environment into SIARD format is started, no modification of the data is needed (de-normalization, additional views, etc.). This is important when we want to maintain data integrity. As for the next step, when a database package is being ingested by the archives, an expert needs to validate it. During the process, queries are manually prepared to show the content of tables, and also serve as proof that the information is well understood and described in the archives. The queries are stored in long-term storage as an access package for a related database AIP and thus enable future users to search the database using typical (parameterized) searches. Finally, when access is requested by an end-user, the archivist deploys the archived database and the access package. Now the user can use a menu to access the information without the assistance of an expert.
+:- representation information that supplements original database data packages,
+:- automatic deployment of database packages (data objects) and related access packages, including redaction
+:- access mechanism to the database via a browser menu with report selection and search screen for a selected database that mimics the original application.
 ====Licensing and cost====
 Free/open-source EUPL-1.2
 ====Development activity====
-dbDIPview has been initially created in 2009 and is often being improved to cover requirements by new use cases.
+dbDIPview has been initially created in 2009 and is continuously being improved to cover requirements by new use cases in the area of long‐term preservation of databases.
 ====Platform and interoperability====
 Linux, PHP, Apache2, PostgreSQL
 ====Functional notes====
-The queries can be prepared using the information from the creator, using some dedicated query builder, or simply manually. When a basic query is ready, the search criteria screen and drill down behaviour can be added. A typical use case assumes relatively simple relational database data and excludes direct data visualization.
+Before export from the original environment is started, no modification of the data is needed, like de-normalization or the creation of additional views. This is important to ensure data integrity and authenticity.
-The ordering process in the archives will result in packages, transferred to the dedicated dbDIPview server. There a database can be deployed and its access package activated. The user can be redirected via a unique code directly to the access menu for the desired database. When access is finished, the database can be immediately removed, or it remains available for other users among other active databases.
-The access menu lists the available reports. For each report, a search screen is configured for entering the search criteria, including drop-down fields. In the results pane, links to further reports can be available from a value in a certain column.
+A database can be stored in SIARD or dbDIPview packaging (based on CSV) format. Complex combinations of data packages are possible.
+In the SIP/AIP Audit Phase in the archives, an expert needs to validate the content. For the archivist, the most convenient way is to use a dissemination package. During the validation process, database queries are prepared to show the content of the tables in the same way it is shown in the reports from the original environment. The queries are written in pure (portable) SQL and are stored in long-term storage as part of the Representation Information Package. The queries can be prepared using the information provided by the creator, using some dedicated query builder, or simply manually. When a basic query is ready, the search criteria panel including drop-down fields, and drill down behaviour can be configured. The test serves as proof that the information in the database is well understood, described for future use and is ready for dissemination.
+Upon request by an end-user, the order fulfillment process enables that packages are transferred to the dedicated dbDIPview server. There, the information in the database Representation Information Package enables seamless automated deployment of data package(s) of the archived database data objects and setting up of the rendering module. Automated database redaction and creation of additional views are possible at this stage. The user can be redirected via a unique code directly to the menu with queries for the desired database.
+When dissemination is not needed anymore, the database can be removed, or it remains available as persistent DIP (DIPp).
+Some benefits of this approach:
+* The quality of ingested data in the SIP is tested during the pre-ingest and ingest process when we prepare the rendering information
+* After the ingest, no further expert involvement is needed for future use of the content because the tool is archival personnel friendly and allows unattended deployment
+* The configurable interface enables mimicking the reporting part of the original application
+* Links to external files are possible from column data
+* Table content and external files can be added to existing DIPp, thus eliminating the need for redeployment.
+* Database restoration can be done from SIARD files (via external tools SIARD Suite or DBPTK) or packages with CSV files
+* Redaction of database tables can be configured as part of representation information to be carried out during deployment
+* Preconfigured end-user friendly browser user interface
+* The end-user can download the result of a query as a CSV file
+* It is possible to combine and link the results, i.e., jump from one report to another one with detailed information
+* Database column descriptions are shown as infotip
+* Simple technology minimizes the dependencies and increases maintainability, technical robustness, and supportability for long-term use
+* PHP code simplifies security audit if it is requested before deployment into the archival environment
 ====Documentation and user support====
-A demonstration example is included with a sample database and related viewer package.
+Demonstration examples are included with sample databases and related Representation Information.
 ====Usability====
-End-users interact with the system through a browser user interface. Users can work in parallel on the same server on any of the accessible databases.
+End-users interact with the system through a browser user interface and work with the accessible databases. The hierarchical reports menu enables selection between available reports to access the information without the assistance of an expert. In the results pane, drill-down links to further reports may be available based on column values. BLOB data and external files may be shown. Additional metadata can be shown with each report to provide a detailed description of the given report.
 ====Expertise required====
-Basic knowledge of Linux system administration is needed for initial installation and configuration. Familiarity with basic SQL commands and understanding of database structure is needed to configure the viewer. No special skills are required for the end-user.
+Basic knowledge of Linux system administration is needed for initial installation and configuration. Familiarity with basic SQL commands and an understanding of database structure is needed to configure the viewer. No special skills are required for the end-user.
 ====Standards compliance====
-dbDIPview uses its XML schema for storing information about the queries, and its format for packaging the access package and optional CSV package. The queries should not use any SQL dialect to minimize compatibility issues with a target database.
+dbDIPview uses its XML schema for storing information about the queries, and its format for packaging the Representation Information Package and optional CSV package. The queries should not use any SQL dialect to minimize compatibility issues with a target database.
 ====Influence and take-up====
-Information about wider use is unavailable.
+The tool is in production use.
 == User Experiences ==
 <!-- Add hotlinks to user experiences with the tool (eg. blog posts). These should illustrate the effectiveness (or otherwise) of the tool. Use a bullet list. -->
+Some details of database data rendering: https://github.com/dbdipview/dbdipview/wiki/Screenshots
 == Development Activity ==
 <!-- Provide *evidence* of development activity of the tool. For example, RSS feeds for code issues or commits. -->
-All development activity is visible on GitHub: https://github.com/dbdipview/dbdipview/wiki
+The tool is stable but new features are added from time to time. All development activity is visible on GitHub.
-{{Infobox_tool_details
-|releases_rss=
-|issues_rss=
-|mailing_lists=
-|ohloh_id=
-}}