Loop/Test/Stack Check

From MozillaWiki
< Loop‎ | Test
Jump to: navigation, search


Summary

This wiki is intended as a guideline to:

  • verify that the correct release has been deployed
  • verify that basic services are up-and-running


Many of the services in the Loop ecosystem are essentially nginx servers and can be verified using similar techniques. i.e. If you've verified one service in any given environment (DEV, STAGE or PRODUCTION), in most cases, you can just change the URL and the verification process will be roughly the same.


The tests listed below are mostly self-explanatory.

#toc


URLS

You want these:

loop-server DEV http://loop-dev.stage.mozaws.net
STAGE http://loop.stage.mozaws.net
PRODUCTION http://loop.services.mozilla.com
loop-client DEV http://loop-webapp.dev.mozaws.net
STAGE http://call.stage.mozaws.net
PRODUCTION http://hello.firefox.com
msisdn-server DEV http://msisdn-dev.stage.mozaws.net
STAGE http://msisdn.stage.mozaws.net
http://msisdn-loadtest.stage.mozaws.net
PRODUCTION http://msisdn.services.mozilla.com


#toc


PRODUCTION Stack Check

Verifying a PRODUCTION deployment can be done in phases to reduce risk. Also, unlike DEV and STAGE environments, the PRODUCTION environment is restricted to Ops team so, verifying server version directly on server may not be possible. A few extra steps are outlined here to ensure a smooth PROD rollout.


Prior to DNS switch (and once the new PROD stack has been deployed), you can modify your /etc/hosts file locally and pointing it to the IP of the incoming service. NOTE: make sure to flush your cache before proceeding or you may still be pointing to the original address.


To ensure DNS has switched from outgoing stack to the newly-deployed one, ping or dig the PRODUCTION URL both before and after DNS switch to verify the change. Once you're sure that DNS has propagated, you can proceed to do final verification.


Summary of Steps

  1. Run dig in terminal against existing PRODUCTION stack (to determine IP)
  2. Ops deploys new PRODUCTION stack
  3. Modify /etc/hosts (on your local machine) to point to IP address of new stack
  4. Flush cache
  5. Do (pre-DNS-switch) verification on incoming stack (see steps outlined below)
  6. Remove record from /etc/hosts (on your local machine)
  7. Flush cache
  8. Ops switches DNS
  9. Run dig again until IP address of PRODUCTION URL changes
  10. Now you're ready to verify the new stack! (see steps outlined below)

#toc


loop-server

STAGE

In Terminal (on STAGE server):

STAGE STACK - PACKAGE VERSION

$ sed -n '/version/p' /data/loop-server/package.json: 

"version": "0.17.7",

Open following URL in Browser: https://loop.stage.mozaws.net returns: 

{
  "name": "mozilla-loop-server",
"description": "The Mozilla Loop (WebRTC App) server",
  "version": "0.17.7", 
  "homepage": "https://github.com/mozilla-services/loop-server/",
  "endpoint": "https://loop.stage.mozaws.net",
  "fakeTokBox": false,
  "fxaOAuth": true,
  "i18n": {
    "defaultLang" : "en-US", 
  "lang": "en-US" 
  }
}


Open following URL in Browser: https://loop.stage.mozaws.net/push-server-config: returns: 

{
  "pushServerURI" : "wss://loop-push1.stage.mozaws.net/"
}


Open following URL in Browser: https://loop.stage.mozaws.net/__heartbeat__ returns: 

{
  "storage" : true,
  "provider": true
}


In Terminal: $ curl -I https://loop.stage.mozaws.net 

HTTP/1.1 200 OK
Date: Mon, 27 Apr 2015 19:58:43 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 273
Connection: keep-alive
Vary: Origin
ETag: W/"111-e944a68d"
Timestamp: 1430164723

#toc


PRODUCTION

Pre-requisite #PRODUCTION deployment verification


Open following URL in Browser: https://loop.services.mozilla.com returns: 

{
  "name": "mozilla-loop-server",
"description": "The Mozilla Loop (WebRTC App) server",
  "version": "0.17.7", 
  "homepage": "https://github.com/mozilla-services/loop-server/",
  "endpoint": "https://loop.services.mozilla.com",
  "fakeTokBox": false,
  "fxaOAuth": true,
  "i18n": {
    "defaultLang" : "en-US", 
  "lang": "en-US" 
  }
}

Open following URL in Browser: https://loop.services.mozilla.com/push-server-config: returns: 

{
  "pushServerURI" : "wss://loop-push1.services.mozilla.com/"
}

Open following URL in Browser: https://loop.services.mozilla.com/__heartbeat__ returns: 

{
  "storage" : true,
  "provider": true
}

In Terminal: $ curl -I https://loop.services.mozilla.com 

HTTP/1.1 200 OK
Date: Mon, 27 Apr 2015 19:58:43 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 273
Connection: keep-alive
Vary: Origin
ETag: W/"111-e944a68d"
Timestamp: 1430164723

#toc

loop-client

STAGE

Open following URL in browser: http(s)://call.stage.mozaws.net/: redirects to: https://www.mozilla.org/en-US/firefox/hello/


Open following URL in browser: https://call.stage.mozaws.net/config.js 

var loop = loop || {};

loop.config = {
  serverUrl: 'https://loop.stage.mozaws.net/v0',
  feedbackApiUrl: 'https://input.allizom.org/api/v1/feedback',
  feedbackProductName: 'Loop',
  downloadFirefoxUrl: 'https://www.mozilla.org/firefox/new/?scene=2&utm_source=hello.firefox.com&utm_medium=referral&utm_campaign=non-webrtc-browser\#download-fx',
  privacyWebsiteUrl: 'https://www.mozilla.org/privacy/firefox-hello/',
  legalWebsiteUrl: 'https://www.mozilla.org/about/legal/terms/firefox-hello/',
  marketplaceUrl: 'https://marketplace-dev.allizom.org/iframe-install.html',
  learnMoreUrl: 'https://www.mozilla.org/hello/',
  roomsSupportUrl: 'https://support.mozilla.org/kb/group-conversations-firefox-hello-webrtc',
  guestSupportUrl: 'https://support.mozilla.org/kb/respond-firefox-hello-invitation-guest-mode',
  generalSupportUrl: 'https://support.mozilla.org/kb/respond-firefox-hello-invitation-guest-mode',
  unsupportedPlatformUrl: 'https://support.mozilla.org/kb/which-browsers-will-work-firefox-hello-video-chat',
  fxosApp: {
    name: 'Hello Stage',
    manifestUrl: 'https://marketplace-dev.allizom.org/app/ae2af828-21be-4483-ae94-476fe48c6cf8/manifest.webapp',
    rooms: true
  }
};


FTU (First-Time User)

  • Click on the "Terms of Use" hyperlink to make sure this has been properly routed
  • Click on the "Privacy Notice" hyperlink to make sure this has been properly routed


In Terminal:

$ curl -I https://call.stage.mozaws.net 

HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 7233
Content-Security-Policy: frame-ancestors 'self'
Content-Type: text/html
Date: Thu, 30 Apr 2015 02:28:16 GMT
ETag: "553f6072-1c41"
Last-Modified: Tue, 28 Apr 2015 10:26:58 GMT
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-Token-Match: 1
Connection: keep-alive


Open the following URL in Browser: https://loop.stage.mozaws.net/: 

{
  "name": "mozilla-loop-server",
  "description": "The Mozilla Loop (WebRTC App) server",
  "version": "0.17.7",
  "homepage": "https://github.com/mozilla-services/loop-server/",
  "endpoint": "https://loop.stage.mozaws.net",
  "fakeTokBox": false,
  "fxaOAuth": true,
  "i18n": {
    "defaultLang": "en-US",
    "lang": "en-US"
  }
}


#toc


PRODUCTION

Pre-requisite #PRODUCTION Stack Check


Open following URL in browser: https://hello.firefox.com/: 

Welcome to the Firefox Hello web client.


Open the following URL in browser: https://hello.firefox.com/config.js 

var loop = loop || {};

loop.config = {
  serverUrl: 'https://loop.services.mozilla.com/v0',
  feedbackApiUrl: 'https://input.mozilla.org/api/v1/feedback',
  feedbackProductName: 'Loop',
  downloadFirefoxUrl: 'https://www.mozilla.org/firefox/new/?scene=2&utm_source=hello.firefox.com&utm_medium=referral&utm_campaign=non-webrtc-browser\#download-fx',
  privacyWebsiteUrl: 'https://www.mozilla.org/privacy/firefox-hello/',
  legalWebsiteUrl: 'https://www.mozilla.org/about/legal/terms/firefox-hello/',
  marketplaceUrl: 'https://marketplace.firefox.com/iframe-install.html',
  learnMoreUrl: 'https://www.mozilla.org/hello/',
  roomsSupportUrl: 'https://support.mozilla.org/kb/group-conversations-firefox-hello-webrtc',
  guestSupportUrl: 'https://support.mozilla.org/kb/respond-firefox-hello-invitation-guest-mode',
  generalSupportUrl: 'https://support.mozilla.org/kb/respond-firefox-hello-invitation-guest-mode',
  unsupportedPlatformUrl: 'https://support.mozilla.org/kb/which-browsers-will-work-firefox-hello-video-chat',
  fxosApp: {
    name: 'Firefox Hello',
    manifestUrl: 'https://marketplace.firefox.com/app/54b83aea-1208-4605-82eb-22819f39d81d/manifest.webapp',
    rooms: true
  }
};


FTU (First-Time User)

  • Click on the "Terms of Use" hyperlink to make sure this has been properly routed
  • Click on the "Privacy Notice" hyperlink to make sure this has been properly routed


In Terminal:

$ curl -I https://hello.firefox.com 

HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 7233
Content-Security-Policy: frame-ancestors 'self'
Content-Type: text/html
Date: Thu, 30 Apr 2015 02:28:16 GMT
ETag: "553f6072-1c41"
Last-Modified: Tue, 28 Apr 2015 10:26:58 GMT
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
X-Token-Match: 1
Connection: keep-alive


Open the following URL in Browser: https://loop.services.mozilla.com/: 

{
  "name": "mozilla-loop-server",
  "description": "The Mozilla Loop (WebRTC App) server",
  "version": "0.17.7",
  "homepage": "https://github.com/mozilla-services/loop-server/",
  "endpoint": "https://loop.services.mozilla.com",
  "fakeTokBox": false,
  "fxaOAuth": true,
  "i18n": {
    "defaultLang": "en-US",
    "lang": "en-US"
  }
}


#toc


msisdn-gateway

STAGE

In Terminal (on STAGE server):

STAGE STACK - PACKAGE VERSION

$ sed -n '/version/p' /data/msisdn-gateway/package.json:

"version": "0.6.0",


Open following URL in browser:

https://msisdn.stage.mozaws.net 

{
  "name": "mozilla-msisdn-gateway",
  "description": "The Mozilla MSISDN Gateway",
  "version": "0.6.0",
  "homepage": "https://github.com/mozilla-services/msisdn-gateway/",
  "endpoint": "https://msisdn.stage.mozaws.net"
}


Open following URL in browser:

https://msisdn-loadtest.stage.mozaws.net 

{
  "name": "mozilla-msisdn-gateway",
  "description": "The Mozilla MSISDN Gateway",
  "version": "0.6.0",
  "homepage": "https://github.com/mozilla-services/msisdn-gateway/",
  "endpoint": "https://msisdn-loadtest.stage.mozaws.net"
}


Open following URL in browser: https://msisdn.stage.mozaws.net/__heartbeat__ 

{
  "storage": true
}


Open following URL in browser: https://msisdn-loadtest.stage.mozaws.net/__heartbeat__ 

{
  "storage": true
}


In Terminal: $ curl -I https://msisdn.stage.mozaws.net 

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Content-Length: 204
Content-Type: application/json; charset=utf-8
Date: Mon, 18 May 2015 22:13:21 GMT
ETag: W/"cc-304f62d"
Timestamp: 1431987201
Connection: keep-alive


In Terminal:

$ curl -I https://msisdn-loadtest.stage.mozaws.net 

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Content-Length: 213
Content-Type: application/json; charset=utf-8
Date: Mon, 18 May 2015 22:13:29 GMT
ETag: W/"d5-c9ef74d9"
Timestamp: 1431987209
Connection: keep-alive


#toc


PRODUCTION

Pre-requisite #PRODUCTION Stack Check


Open following URL in browser:

https://msisdn.services.mozilla.com 

{
  "name": "mozilla-msisdn-gateway",
  "description": "The Mozilla MSISDN Gateway",
  "version": "0.6.0",
  "homepage": "https://github.com/mozilla-services/msisdn-gateway/",
  "endpoint": "https://msisdn.services.mozilla.com"
}


Open following URL in browser:

https://msisdn.services.mozilla.com/__heartbeat__ 

{
  "storage": true
}


In Terminal: $ curl -I https://msisdn.services.mozilla.com 

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Content-Length: 204
Content-Type: application/json; charset=utf-8
Date: Mon, 18 May 2015 22:13:21 GMT
ETag: W/"cc-304f62d"
Timestamp: 1431987201
Connection: keep-alive

In Terminal:


#toc


File System (Configs, Logs, etc.)

Files

  • /opt/aws
  • /opt/ec2
  • /opt/openresty
  • /opt/rh
  • /opt/stackdriver
  • /data/hekad
  • /data/msisdn-gateway
  • /etc/puppet/yaml/app, especially
    • msisdn_gateway.dev.yaml
    • msisdn_gateway.prod.yaml
    • msisdn_gateway.stage.yaml
    • msisdn_gateway.yaml


Processes

  • Look for stackdriver, python, circus, resty, nginx, node, heka


Logs

  • /media/ephemeral0/msisdn-gateway/msisdn-gateway_err.log
  • /media/ephemeral0/msisdn-gateway/msisdn-gateway_out.log
  • /media/ephemeral0/nginx/logs/default.access.log (not in use)
  • /media/ephemeral0/nginx/logs/default.error.log (not in use)
  • /media/ephemeral0/nginx/logs/msisdn-gateway.access.log
  • /media/ephemeral0/nginx/logs/msisdn-gateway.error.log
  • /var/log/circus.log
  • /var/log/hekad/msisdn_gateway.stderr.log
  • /var/log/hekad/msisdn_gateway.stdout.log


Configuration File

  • /data/msisdn-gateway/config/production.json


#toc

Retrieved from "https://wiki.mozilla.org/index.php?title=Loop/Test/Stack_Check&oldid=1077779"