Duke Wiki  logo
Skip to end of metadata
Go to start of metadata

Notes from 20161022: Pipeline issue I worked on: Updated the Open Access URL exclusion list for ServiceNow ticket TASK1600056: “Endeca "open access" resource links”. Derrek submitted the code info Endeca SVN, but I did not copy the code into serverC. It worked the first week (after sending the bib number through Aleph’s manual update process), but the next week it no longer worked correctly (i.e., the catalog displayed “Open Access” in the URL). Jeff noted that Sunday’s baseline run probably set the Pipeline code back to old version, so now I've copied Duke’s pipeline dir, creating a backup and edited the (new) current Pipeline.epx file to have the correct changes.

Step-by-step guide

  1. Make sure you have the correct URL to be added to OA exclusions list.

  2. Get and/or check the latest changes made to pipeline in SVN, in Endeca serverC:

    svn log svn+ssh://endeca@trln-dev.trln.org/data/svn-repos/pipeline | less

  3. Diff the latest change in SVN:
    svn diff svn+ssh://endeca@trln-dev.trln.org/data/svn-repos/pipeline -r 4783

  4. Make the code change and submit to SVN...

  5. Copy the code into server-C, where the pipelines are.

    In /data/DUKE/config: [endeca@trlnr610c config]$ diff pipeline.20161022.ayse_openAccessURL pipeline

    diff pipeline.20161022.ayse_openAccessURL/Pipeline.epx pipeline/Pipeline.epx
    <     <PASS_THROUGH NAME="EXCLUDE_URL_TEXT">catdir|metasearch|journals\.uchicago|humanities\.uchicago|muse\.edu|getitatduke|duke|\.com|ARTFL|http://stephanus\.tlg\.uci\.edu|ropercenter\.uconn\.edu|vha\.usc\.edu|bmc\.lib\.umich\.edu|rotunda\.upress\.virginia\.edu|quod\.lib\.umich\.edu|muse\.jhu\.edu|press\.jhu\.edu|ica1\.princeton\.edu|hapi\.gseis\.ucla\.edu|ets\.umdl\.umich\.edu|ehrafworldcultures\.yale\.edu|ccrd\.usc\.cuhk\.edu\.hk|bna\.birds\.cornell\.edu</PASS_THROUGH>
    >     <PASS_THROUGH NAME="EXCLUDE_URL_TEXT">catdir|metasearch|journals\.uchicago|humanities\.uchicago|muse\.edu|getitatduke|duke|\.com|ARTFL|http://stephanus\.tlg\.uci\.edu|ropercenter\.uconn\.edu|vha\.usc\.edu|bmc\.lib\.umich\.edu|rotunda\.upress\.virginia\.edu|quod\.lib\.umich\.edu|muse\.jhu\.edu|press\.jhu\.edu|ica1\.princeton\.edu|hapi\.gseis\.ucla\.edu|ets\.umdl\.umich\.edu|ehrafworldcultures\.yale\.edu|ccrd\.usc\.cuhk\.edu\.hk|bna\.birds\.cornell\.edu|woolf-center\.southernct\.edu</PASS_THROUGH>

  6. Add the affected bib record to the aleph-to-endeca manual update id list.