Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add 3d objects metadata block #11167

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from

Conversation

stevenwinship
Copy link
Contributor

@stevenwinship stevenwinship commented Jan 17, 2025

What this PR does / why we need it: Add to the metadata blocks that ship with Dataverse a metadata block for describing 3D data

Which issue(s) this PR closes: #11120

Special notes for your reviewer:

Suggestions on how to test this: create dataverse with added 3d metadata block. Create new dataset. add 3d object block and test searching with those fields

Does this PR introduce a user interface change? If mockups are available, please link/include them here: No

Is there a release notes update needed for this change?: Included

Additional documentation:

@stevenwinship stevenwinship self-assigned this Jan 17, 2025
@stevenwinship stevenwinship added Type: Feature a feature request Feature: Metadata Size: 10 A percentage of a sprint. 7 hours. FY25 Sprint 15 FY25 Sprint 15 (2025-01-15 - 2025-01-29) labels Jan 17, 2025
Copy link

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:11120-add-3d-object-metadata-block
ghcr.io/gdcc/configbaker:11120-add-3d-object-metadata-block

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.

@coveralls
Copy link

Coverage Status

coverage: 22.751%. remained the same
when pulling 4b94bf5 on 11120-add-3d-object-metadata-block
into a6e36f6 on develop.

@stevenwinship stevenwinship removed their assignment Jan 17, 2025
@pdurbin pdurbin changed the title added 3d objects metadata block add 3d objects metadata block Jan 22, 2025
@pdurbin pdurbin self-assigned this Jan 24, 2025
Copy link
Member

@pdurbin pdurbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some feedback.

3dHeight Height The rough height of the object text 11 Height: #VALUE FALSE FALSE FALSE FALSE FALSE FALSE 3dObjectDimensions 3dobjects
3dWeight Weight The rough weight of the object text 12 Weight:#VALUE FALSE FALSE FALSE FALSE FALSE FALSE 3dObjectDimensions 3dobjects
3dUnit Unit The unit of measurement used for the object dimensions text 13 Unit: #VALUE FALSE TRUE FALSE TRUE FALSE FALSE 3dObjectDimensions 3dobjects
3dHandling Instructions Saftey and special handling instructions for the object textbox 14 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE 3dobjects
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
3dHandling Instructions Saftey and special handling instructions for the object textbox 14 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE 3dobjects
3dHandling Instructions Safety and special handling instructions for the object textbox 14 #VALUE FALSE FALSE FALSE FALSE FALSE FALSE 3dobjects

datasetfieldtype.3dHeight.description=The rough height of the object
datasetfieldtype.3dWeight.description=The rough weight of the object
datasetfieldtype.3dUnit.description=The unit of measurement used for the object dimensions
datasetfieldtype.3dHandling.description=Saftey and special handling instructions for the object
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
datasetfieldtype.3dHandling.description=Saftey and special handling instructions for the object
datasetfieldtype.3dHandling.description=Safety and special handling instructions for the object

@@ -30,6 +30,7 @@ Detailed below are what metadata schemas we support for Citation and Domain Spec
`Virtual Observatory (VO) Discovery and Provenance Metadata <https://perma.cc/H5ZJ-4KKY>`__.
- Life Sciences Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/biomedical.tsv>`__): based on `ISA-Tab Specification <https://isa-specs.readthedocs.io/en/latest/isamodel.html>`__, along with controlled vocabulary from subsets of the `OBI Ontology <https://bioportal.bioontology.org/ontologies/OBI>`__ and the `NCBI Taxonomy for Organisms <https://www.ncbi.nlm.nih.gov/Taxonomy/taxonomyhome.html/>`__.
- Journal Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/journals.tsv>`__): based on the `Journal Archiving and Interchange Tag Set, version 1.2 <https://jats.nlm.nih.gov/archiving/tag-library/1.2/chapter/how-to-read.html>`__.
- 3D Objects Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/3d_objects.tsv>`__) __.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- 3D Objects Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/3d_objects.tsv>`__) __.
- 3D Objects Metadata (`see .tsv <https://github.com/IQSS/dataverse/blob/master/scripts/api/data/metadatablocks/3d_objects.tsv>`__).

I didn't test this but I see a weird underscore at https://dataverse-guide--11167.org.readthedocs.build/en/11167/user/appendix.html

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this update-datasetfields.sh used anywhere? I don't think so, based on the history: https://github.com/IQSS/dataverse/commits/4b94bf56aed2b1f5aae2762cd6a454964bed6bbe/scripts/api/update-datasetfields.sh

Should we simply delete this script? @sekmiller you added it in 1787fc7. Are you (or anyone reading this) actively using it?

Comment on lines +1 to +3
This feature adds a new Metadata block for describing 3D Object Data

The new Metadata block will be added during the installation or upgrade of Dataverse
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This feature adds a new Metadata block for describing 3D Object Data
The new Metadata block will be added during the installation or upgrade of Dataverse
### New 3D Object Data Metadata Block
A new metadata block has been added for describing 3D object data. You can download it from the [guides](https://dataverse-guide--11167.org.readthedocs.build/en/11167/user/appendix.html). See also #11120 and #11167.
All new Dataverse installations will receive this metadata block by default. We recommend adding it by following the upgrade instructions below.
## Upgrade Instructions
TODO: local 3d block, update schema, etc.

As I wrote below, I don't think update-datasetfields.sh is actively used and I'm not sure why we're maintaining it. There's no magic to do all the updating (especially including the Solr schema) so we have to write it out. Some suggestions are below.

In a GitHub suggestion I can't add fenced code blocks so I'm just going to post verbatim the 6.4 part I think we need to add and tweak. We need to change citation to 3dwhatever and otherwise edit this and add it where I'm putting TODO:

6. Update metadata blocks

These changes reflect incremental improvements made to the handling of core metadata fields.

wget https://raw.githubusercontent.com/IQSS/dataverse/v6.4/scripts/api/data/metadatablocks/citation.tsv

curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file citation.tsv

7. Update Solr schema.xml file. Start with the standard v6.4 schema.xml, then, if your installation uses any custom or experimental metadata blocks, update it to include the extra fields (step 7a).

Stop Solr (usually service solr stop, depending on Solr installation/OS, see the Installation Guide).

sudo service solr stop

Replace schema.xml

wget https://raw.githubusercontent.com/IQSS/dataverse/v6.4/conf/solr/schema.xml
sudo cp schema.xml /usr/local/solr/solr-9.4.1/server/solr/collection1/conf

Start Solr (but if you use any custom metadata blocks, perform the next step, 7a first).

sudo service solr start

7a. For installations with custom or experimental metadata blocks:

Before starting Solr, update the schema to include all the extra metadata fields that your installation uses. We do this by collecting the output of the Dataverse schema API and feeding it to the update-fields.sh script that we supply, as in the example below (modify the command lines as needed to reflect the names of the directories, if different):

	wget https://raw.githubusercontent.com/IQSS/dataverse/v6.4/conf/solr/update-fields.sh
	chmod +x update-fields.sh
	curl "http://localhost:8080/api/admin/index/solr/schema" | sudo ./update-fields.sh /usr/local/solr/solr-9.4.1/server/solr/collection1/conf/schema.xml

Now start Solr.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume we'd get the same results if we use the technique here:

I'm just raising awareness of those new docs about using Docker to consistently sort the fields.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature: Metadata FY25 Sprint 15 FY25 Sprint 15 (2025-01-15 - 2025-01-29) Size: 10 A percentage of a sprint. 7 hours. Type: Feature a feature request
Projects
Status: In Review 🔎
3 participants