Uploaded image for project: 'Blazegraph (by SYSTAP)'
  1. Blazegraph (by SYSTAP)
  2. BLZG-1016

Merge DEPLOYMENTS branch back to the main branch

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Affects Version/s: BIGDATA_RELEASE_1_3_1
    • Fix Version/s: None
    • Component/s: Other

      Description

      Merge down the DEPLOYMENTS branch into the main development branch.

      Close out the slew of tickets related to the DEPLOYMENTS branch.

      Documentation status on unresolved issues (RPMs) and provide sufficient documentation for ongoing maintenance of the various deployers.

      Note: branches/DEPLOYMENT_BRANCH_1_3_1 was created in r8165.

        Activity

        Hide
        bryanthompson bryanthompson added a comment -

        Commit to the deployments branch:


        - jetty.xml: I have replaced all instances of SystemProperty with Property.


        - log4jHA.properties: The main branch has correct log4j configuration examples for jetty.


        - Reformatted HARestore, but did not address the issue with how the classpath is assembled.

        Committed revision r8413 (DEPLOYMENTS branch).

        Show
        bryanthompson bryanthompson added a comment - Commit to the deployments branch: - jetty.xml: I have replaced all instances of SystemProperty with Property. - log4jHA.properties: The main branch has correct log4j configuration examples for jetty. - Reformatted HARestore, but did not address the issue with how the classpath is assembled. Committed revision r8413 (DEPLOYMENTS branch).
        Hide
        bryanthompson bryanthompson added a comment -

        Comments on the deployments branch:

        - HARestore.sh: You can not safely rely on the limited classpath that
          is used in this script.  This is very likely to break based merely
          on the imports into the HARestore, Journal, AbstractJournal and
          related classes.  At a minimum, we would need to test this classpath
          for each release or in CI.  I would prefer that we had a means to
          assemble a better classpath. The startHAServices script has a
          similar problem.  The classpath is currently hacked there using the
          incantation
        
        export HAJOURNAL_CLASSPATH=`find ${LIB_DIR} -name '*.jar' -print0 | tr '\0' ':'`
        
        - bigdataHA script: What is the purpose of the redirect of all output
          into /dev/null?  Is this just a better practice or did you run into
          a specific problem?
        
        old:
        		action $"`date` : `hostname` : bringing up services: " sudo -u $BD_USER -g $BD_GROUP bash -c "source /etc/default/bigdataHA ; $binDir/startHAServices"
        
        new:
        		action $"`date` : `hostname` : bringing up services: " sudo -u $BD_USER -g $BD_GROUP bash -c "source /etc/default/bigdataHA ; $binDir/startHAServices > /dev/null 2>&1 &"
        
        - What is the purpose of the "src/resources/deployment" directory?  Is
          this the "single-server, non-HA" NSS deployment?
        
        Show
        bryanthompson bryanthompson added a comment - Comments on the deployments branch: - HARestore.sh: You can not safely rely on the limited classpath that is used in this script. This is very likely to break based merely on the imports into the HARestore, Journal, AbstractJournal and related classes. At a minimum, we would need to test this classpath for each release or in CI. I would prefer that we had a means to assemble a better classpath. The startHAServices script has a similar problem. The classpath is currently hacked there using the incantation export HAJOURNAL_CLASSPATH=`find ${LIB_DIR} -name '*.jar' -print0 | tr '\0' ':'` - bigdataHA script: What is the purpose of the redirect of all output into /dev/null? Is this just a better practice or did you run into a specific problem? old: action $"`date` : `hostname` : bringing up services: " sudo -u $BD_USER -g $BD_GROUP bash -c "source /etc/default/bigdataHA ; $binDir/startHAServices" new: action $"`date` : `hostname` : bringing up services: " sudo -u $BD_USER -g $BD_GROUP bash -c "source /etc/default/bigdataHA ; $binDir/startHAServices > /dev/null 2>&1 &" - What is the purpose of the "src/resources/deployment" directory? Is this the "single-server, non-HA" NSS deployment?
        Hide
        bryanthompson bryanthompson added a comment -


        - HARestore.sh: You can not safely rely on the limited classpath that

        is used in this script. This is very likely to break based merely

        on the imports into the HARestore, Journal, AbstractJournal and

        related classes. At a minimum, we would need to test this classpath

        for each release or in CI. I would prefer that we had a means to

        assemble a better classpath. The startHAServices script has a

        similar problem. The classpath is currently hacked there using the

        incantation

        export HAJOURNAL_CLASSPATH=find ${LIB_DIR} -name '*.jar' -print0 | tr '\0' ':'


        - What is the purpose of the "src/resources/deployment" directory? Is

        this the "single-server, non-HA" NSS deployment?


        - /bigdata/deployment
        - we put all of this stuff under /src/resources NOT /bigdata.


        - I have deleted /bigdata/deployment entirely from branches/BIGDATA_RELEASE_1_3_0.


        - I have copied the files (but not the SVN folders) from the

        DEPLOYMENT_BRANCH_1_3_1/bigdata/src/resources/deployment into
        /src/resources/deployment.


        - jetty.xml: copied from the DEPLOYMENTS branch.


        - /Users/bryan/Documents/workspace/BIGDATA_RELEASE_1_3_0/src/resources/bin/bigdata.sh


        - This has been removed. The src/resources/deployment/nss directory

        has similar scripts. It is Ok to add an ant task to start the nss
        for developers, but deployments should be based on the "ant stage"
        pattern.


        - src/resources/deployment/nss/WEB-INF/RWStore.properties should be

        removed. The brew script should replace the following line in the

        version from bigdata-war/src/WEB-INF/RWStore.properties with an

        absolute filename.

        com.bigdata.journal.AbstractJournal.file=ZZZZ
        


        - src/resources/deployment/nss/WEB-INF/log4j.properties should be

        removed. The brew script should replace the following lines in the

        version from dist/var/config/logging/log4j.properties in order to

        setup (a) logging to a file; and (b) to specify the absolution

        location of that file.

        log4j.rootCategory=XXXX
        
        log4j.appender.file.File=YYYY
        


        - build.xml: I have declared variables ${deploy} and ${deploy.nss} for the src/resources/deployments folders.


        - build.xml: These lines were removed per BLZG-1016 (Deployments branch merge). They break the other deployment models by introducing metavariables for regex substitutions.

         bigdata-war/src/WEB-INF/RWStore.properties (staged into bigdata/var/jetty/bigdata/WEB-INF/RWStore.properties)
        

        and

         
         bigdata/src/resources/log4j.properties (staged into dist/var/config/logging/log4j.properties).
            	<copy file="${deploy.nss}/WEB-INF/RWStore.properties"
                    todir="${dist.var.jetty}/WEB-INF" overwrite="true" />
              <copy file="${deploy.nss}/WEB-INF/classes/log4j.properties"
                    todir="${dist.var.jetty}/WEB-INF/classes" overwrite="true" />
        

        Daniel will fix this and recommit.

        Committed revision r8415 (partial commit)
        Committed revision r8415 (fixes a broken commit)

        Show
        bryanthompson bryanthompson added a comment - - HARestore.sh: You can not safely rely on the limited classpath that is used in this script. This is very likely to break based merely on the imports into the HARestore, Journal, AbstractJournal and related classes. At a minimum, we would need to test this classpath for each release or in CI. I would prefer that we had a means to assemble a better classpath. The startHAServices script has a similar problem. The classpath is currently hacked there using the incantation export HAJOURNAL_CLASSPATH= find ${LIB_DIR} -name '*.jar' -print0 | tr '\0' ':' - What is the purpose of the "src/resources/deployment" directory? Is this the "single-server, non-HA" NSS deployment? - /bigdata/deployment - we put all of this stuff under /src/resources NOT /bigdata. - I have deleted /bigdata/deployment entirely from branches/BIGDATA_RELEASE_1_3_0. - I have copied the files (but not the SVN folders) from the DEPLOYMENT_BRANCH_1_3_1/bigdata/src/resources/deployment into /src/resources/deployment. - jetty.xml: copied from the DEPLOYMENTS branch. - /Users/bryan/Documents/workspace/BIGDATA_RELEASE_1_3_0/src/resources/bin/bigdata.sh - This has been removed. The src/resources/deployment/nss directory has similar scripts. It is Ok to add an ant task to start the nss for developers, but deployments should be based on the "ant stage" pattern. - src/resources/deployment/nss/WEB-INF/RWStore.properties should be removed. The brew script should replace the following line in the version from bigdata-war/src/WEB-INF/RWStore.properties with an absolute filename. com.bigdata.journal.AbstractJournal.file=ZZZZ - src/resources/deployment/nss/WEB-INF/log4j.properties should be removed. The brew script should replace the following lines in the version from dist/var/config/logging/log4j.properties in order to setup (a) logging to a file; and (b) to specify the absolution location of that file. log4j.rootCategory=XXXX log4j.appender.file.File=YYYY - build.xml: I have declared variables ${deploy} and ${deploy.nss} for the src/resources/deployments folders. - build.xml: These lines were removed per BLZG-1016 (Deployments branch merge). They break the other deployment models by introducing metavariables for regex substitutions. bigdata-war/src/WEB-INF/RWStore.properties (staged into bigdata/var/jetty/bigdata/WEB-INF/RWStore.properties) and bigdata/src/resources/log4j.properties (staged into dist/var/config/logging/log4j.properties). <copy file="${deploy.nss}/WEB-INF/RWStore.properties" todir="${dist.var.jetty}/WEB-INF" overwrite="true" /> <copy file="${deploy.nss}/WEB-INF/classes/log4j.properties" todir="${dist.var.jetty}/WEB-INF/classes" overwrite="true" /> Daniel will fix this and recommit. Committed revision r8415 (partial commit) Committed revision r8415 (fixes a broken commit)

          People

          • Assignee:
            bryanthompson bryanthompson
            Reporter:
            bryanthompson bryanthompson
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: