Initial commit of dms3-docs.

.gitignore

+site/

README.md

+
+# to support diagrams in markdown documents, add atom package:
+#       "Markdown Preview Enhanced"
+# use mkdocs to generate documentation site using markdown.
+#
+# mkdocs setup
+# references:
+# https://www.mkdocs.org/
+# https://www.mkdocs.org/user-guide/configuration/
+# https://github.com/mkdocs/mkdocs/
+# https://github.com/shd101wyy/markdown-preview-enhanced/blob/master/docs/diagrams.md
+#
+# use mkdocs pymdown-extensions with superfences and custom fences to
+# support diagrams in mkdocs.
+# https://facelessuser.github.io/pymdown-extensions/
+# https://facelessuser.github.io/pymdown-extensions/extensions/superfences/
+# https://mermaidjs.github.io/#/
+# https://github.com/squidfunk/mkdocs-material/issues/693
+# https://cdnjs.com/
+# https://github.com/mermaid-js/mermaid/
+
+mkdir -p ~/Work/dapp/dms3
+mkdir -p ~/Work/dapp/dms3/mkdocs-docs
+cd ~/Work/dapp/dms3
+
+#
+#Note: mermaid was removed after the mkdocs documentation was functional
+#initially the docs folder was setup as ~/Work/dapp/dms3/mkdocs-docs,
+#but later moved to ~/Work/dapp/dms3-docs
+#
+#Note: use clone of repo mermaid - cdnjs version has bad css that when a
+# style is added to flowchart svg, the page is disabled.
+# to build mermaid locally after cloning repo:
+git clone https://github.com/mermaid-js/mermaid.git
+cd mermaid/
+nvm list
+nvm use stable
+npm install webpack
+npm install yarn
+npm run build
+npm run release
+ls -alg dist/
+cp -v dist/mermaid.min.js* ../mkdocs-docs/docs/js/
+cp -v dist/mermaid.js* ../mkdocs-docs/docs/js/
+
+cd ../
+python --version
+pip --version
+sudo apt-get install pip
+
+wget https://bootstrap.pypa.io/get-pip.py
+sudo python get-pip.py
+
+sudo pip install mkdocs
+mkdocs --version
+mkdocs new mkdocs-docs
+cd mkdocs-docs
+mkdocs serve
+sudo pip install pymdown-extensions
+sudo pip install mkdocs-rtd-dropdown
+
+# to support math rendering:
+# https://stackoverflow.com/questions/27882261/mkdocs-and-mathjax/31926644
+# https://squidfunk.github.io/mkdocs-material/extensions/pymdown/
+# https://facelessuser.github.io/pymdown-extensions/extensions/arithmatex/
+
+# roboto-slab fonts were downloaded from:
+# Free download fonts at http://sharefonts.net
+# unzip and move missing fonts into mkdocs-docs/docs/fonts/
+
+# usage
+cd ~/Work/dapp/dms3-docs
+
+# build
+mkdocs build
+
+# local live reload server
+mkdocs serve # defaults to 127.0.0.1:8000
+mkdocs serve --dev-addr=192.168.0.7:8000

docs/about.md

+
+## About
+
+DMS3 implements a hybrid dual protocol overlay network stack designed for strong privacy and data security.
+
+A decentralized permission-less secure peer to peer (p2p) blockchain network that provides applications (dapp) for dis-intermediated endpoint p2p interactions. All dapp activity and generated data is stored locally on the endpoint node running the application.
+
+A centralized permission-ed network protects personal data and privacy providing centralized services while enabling broad competition. The initial implementation of a centralized service is information storage and retrieval.

docs/arch.md

+
+## Architecture Overview
+
+DMS3 software runs on network *endpoint nodes* [^1] and consist of thin decentralized client applications or *dapps* [^2] and a thick *network protocol stack* [^3] that includes *dapp-consistent fault tolerant information and data persistence* services [^4] as shown in **Fig. 1: Architecture Overview**.
+
+[^1]: endpoint nodes are compute and storage nodes of varying capacity that include SmartPhones, Tablets, Laptops, Desktops, and Servers.
+[^2]: dapps are decentralized applications that run locally and store all information locally, i.e. there is no web site involved that may track personal activity. A dapp and the data it generates are strictly under the control of the individual controlling the endpoint.
+[^3]: network protocol stack enables an endpoint to interact with peer endpoints and access technological capability without compromising personal security and privacy. The peer-to-peer (p2p) network protocol network implements an overlay network over the Internet reducing network threat footprint.
+[^4]: fault tolerant data services unburden greenfield application developers from the complexities of implementing data scaling, reliability, and availability.
+
+|![Architecture Overview](img/dms3-1.jpg) |
+|:--:|
+| **Fig. 1: Architecture Overview** |
+
+### DApp
+
+DMS3 provides discovery of, and access to, decentralized applications that run on endpoint nodes. Dapp generated data persists on an endpoint node that runs a dapp.
+
+Dapps may communicate with other compatible dapps via a locally running DMS3 API and Network Protocol Stack as the only intermediary, no web server or other third party intermediaries are involved.
+
+### DMS3 API
+
+The DMS3 API provide a layer of isolation to dapps from the affects of changes in the network layer that implement new capabilities.
+
+### Network Protocol Stack
+
+The network layer consists of two major subsystems, a permission-less decentralized blockchain network for p2p interactions and a centralized permission-ed network for high performance access to scalable information storage and retrieval providing data protection, reliability and availability.
+
+#### P2P Protocol Network
+
+A blockchain network similar to Ethereum.
+
+#### PBFT Storage and Retrieval
+
+Practical Byzantine Fault Tolerant service implementations of Filesystem and Search Engine services. This network provides data hosting services extending personal compute and storage node capacity via centralized network services that protect personal data and privacy without compromising endpoint control over generated data.
+
+### Network Topology
+
+The architecture makes basic assumptions about network topology to enable peer nodes to discover other nodes in the network. The following sections provide diagrams for the various network topology perspectives.
+
+#### Physical Network Topology
+
+The following residential and enterprise [Physical Network Topology](arch/phynet.md) should be adequate for endpoint node connectivity.
+
+#### P2P Logical Network Topology
+
+The following [P2P Logical Network Topology](arch/p2pnet.md) diagram provides a logical view of the permission-less p2p network.
+
+#### Centralized Private Overlay Network
+
+The following [Centralized Private Overlay Network](arch/mynet.md) diagram provides a logical view of the permission-ed PBFT network.
+
+#### Hybrid Overlay Network
+
+The following [Hybrid Overlay Network](arch/hybnet.md) diagram provides a logical view of the hybrid permission-less and permission-ed network.
+
+### Information storage abstractions
+
+Computing systems implement various layers of abstractions to store and
+manage information:
+
+- Bit, Byte or Octet, and Block representation (ex: RAID storage)
+- File representation (ex: btrfs file system)
+- Database and Search Engine representations (ex: postgres SQL database, MongoDB NoSQL database, etc...)
+
+With exponential growth in digitized information, innovations in storage
+systems have evolved to store an ever growing amount of information with
+good performance. While innovations in information retrieval (IR) systems has
+enabled indexing and fast retrieval of textual information.
+
+Substantial applications use databases systems to store and manage data
+abstracted as structured relational data, or unstructured (document) data.
+Retrieval is supported using Structured Query Language (SQL), or other query
+languages in the case of NoSql databases. Database scaling and performance
+suffers when data growth is unbounded. The data stored by these applications
+is not transparent, each application requires embedding a search feature to
+provide access to data it collects.
+
+Search engines offer a very fast retrieval technology for the discovery of
+web page information on the Internet, and for information stored on a desktop
+or within enterprise information systems. Search engines work well with textual
+information but don't support the robust data types needed by applications
+as supported by databases. Search engine performance also suffers when
+data growth is unbounded. Popular web search engines expose a flat view of the
+search information space, it is not possible to guide the search request with
+context or metadata input.
+
+Furthermore, all of the above discussed information storage and retrieval
+systems provide data service guarantees from the storage system perspective.
+Every scalable application working with very large data sets is still
+burdened with providing its own data guarantees. A data storage abstraction
+layer that provides application perspective data service guarantees for all
+cooperating applications can provide great benefits to application developers.
+
+DMS3 is a novel solution that combines information storage and retrieval
+and offers several advantages to applications developed with the DMS3 API:
+
+- Search engine retrieval performance
+- Text and application data type support
+- Bounded information containers for consistent performance
+- Context sensitive storage and search
+- Improved data privacy and access control
+- Application data consistency, reliability, safety, and recovery
+- Consistent API for Internet, desktop, and enterprise search applications
+- Easy information sharing between independent applications

docs/arch/hybnet.md

+
+## Hybrid Overlay Network
+
+<style>
+  svg {
+    width: 100%;
+  }
+</style>
+
+---
+
+```mermaid
+graph TD;
+
+    subgraph id20[Hybrid Overlay Network]
+
+    id3[Server];
+    id4[Desktop];
+    id5[Laptop];
+    id6[Tablet];
+    id7[Smartphone];
+
+    id10((P2P Public, Private Overlay Network));
+
+    id3 --- id10;
+    id4 --- id10;
+    id5 --- id10;
+    id6 -.- id10;
+    id7 -.- id10;
+
+    style id20 color:#222,fill:#aaa,stroke:#000,stroke-width:2px;
+    classDef pnet color:#222,fill:#68f,stroke:#000,stroke-width:2px;
+    classDef mynet color:#222,fill:#f86,stroke:#000,stroke-width:2px;
+    classDef hnet color:#222,fill:#f8a,stroke:#000,stroke-width:2px;
+
+    class id10 hnet;
+    class id3 pnet;
+    class id4 pnet;
+    class id5 pnet;
+    class id6 pnet;
+    class id7 pnet;
+
+    id13[Server];
+    id14[Desktop];
+    id15[Laptop];
+    id16[Tablet];
+    id17[Smartphone];
+
+    id10 --- id13
+    id10 --- id14
+    id10 --- id15
+    id10 --- id16
+    id10 --- id17
+
+    class id13 mynet;
+    class id14 mynet;
+    class id15 mynet;
+    class id16 mynet;
+    class id17 mynet;
+
+    end
+
+```
+
+---

docs/arch/mynet.md

+
+## Centralized Private Overlay Network
+
+<style>
+  svg {
+    width: 100%;
+  }
+</style>
+
+---
+
+```mermaid
+graph TD;
+
+    id10((P2P Overlay Network));
+
+    style id20 color:#222,fill:#aaa,stroke:#000,stroke-width:2px;
+
+    classDef mynet color:#222,fill:#f86,stroke:#000,stroke-width:2px;
+
+    class id10 pnet;
+    class id3 pnet;
+    class id4 pnet;
+    class id5 pnet;
+    class id6 pnet;
+    class id7 pnet;
+
+    subgraph id20[Centralized Private Network]
+
+    class id20 pnet;
+
+    id13[Server];
+    id14[Desktop];
+    id15[Laptop];
+    id16[Tablet];
+    id17[Smartphone];
+
+    id10 --- id13
+    id10 --- id14
+    id10 --- id15
+    id10 --- id16
+    id10 --- id17
+
+    class id10 mynet;
+    class id13 mynet;
+    class id14 mynet;
+    class id15 mynet;
+    class id16 mynet;
+    class id17 mynet;
+
+    end
+
+```
+
+---

docs/arch/p2pnet.md

+
+## P2P Logical Network Topology
+
+<style>
+  svg {
+    width: 100%;
+  }
+</style>
+
+---
+
+```mermaid
+graph TD;
+
+    classDef pnet color:#222,fill:#68f,stroke:#000,stroke-width:2px;
+
+    style id20 color:#222,fill:#aaa,stroke:#000,stroke-width:2px;
+
+    subgraph id20[P2P Logical Network]
+
+    id3[Server];
+    id4[Desktop];
+    id5[Laptop];
+    id6[Tablet];
+    id7[Smartphone];
+
+    id10((P2P Overlay Network));
+
+    id3 --- id10;
+    id4 --- id10;
+    id5 --- id10;
+    id6 --- id10;
+    id7 --- id10;
+
+    class id10 pnet;
+    class id3 pnet;
+    class id4 pnet;
+    class id5 pnet;
+    class id6 pnet;
+    class id7 pnet;
+
+    end
+
+```
+
+---

docs/arch/phynet.md

+
+## Physical Network Topology
+
+<style>
+  svg {
+    width: 100%;
+  }
+</style>
+
+---
+
+#### Residential Network
+
+```mermaid
+graph TD;
+
+    id0((Internet))
+    id0 --- id1;
+
+    id1{{Internet Service Provide Metropolitan Network}};
+    id1 --- id2;
+
+    subgraph sid1[Residential Building]
+    id2[/Router\];
+
+    id3[Server];
+    id4[Desktop];
+    id5[Laptop];
+    id6[Tablet];
+    id7[Smartphone];
+
+    id2 --- id3;
+    id2 --- id4;
+    id2 --- id5;
+    id2 --- id6;
+    id2 --- id7;
+    end
+
+    style sid1 color:#222,fill:#68f,stroke:#000,stroke-width:2px
+
+```
+
+---
+
+#### Enterprise Network
+
+
+```mermaid
+graph TD;
+
+    id0((Internet))
+    id0 --- id1;
+
+    id1{{Internet Service Provide Metropolitan Network}};
+
+    id1 --- id20;
+
+    subgraph sid2[Enterprise Building]
+    id20[/Router 1\];
+
+    subgraph sid3[Data Center]
+    id21[Server];
+    id22[Server];
+    id23[Server];
+    end
+
+    id24[Desktop];
+    id25[Laptop];
+    id26[Tablet];
+    id27[Smartphone];
+
+    id20 --- id21;
+    id20 --- id22;
+    id20 --- id23;
+    id20 --- id24;
+    id20 --- id25;
+    id20 --- id26;
+    id20 --- id27;
+    end;
+
+    style sid2 color:#fff,fill:#68f,stroke:#000,stroke-width:2px
+    style sid3 color:#fff,fill:#377,stroke:#000,stroke-width:2px
+
+```
+
+---

docs/how.md

+
+## How is DMS3 Better?
+
+### Informed Data Sharing
+
+- Information is intrinsically stationary
+- No unsolicited information flow, receiving endpoint must be pull or opt-in
+- Each source independently chooses information to share publicly
+- Centralized services are optional
+    - Data distribution capacity, availability, and reliability services
+    - Independent centralized service sources
+
+### Competitive Revenue Sharing Opportunity
+
+- Any source can earn coins
+- Any source can spend coins
+- Any centralized service source can earn coins
+
+### Balanced Personal Protections
+
+- Strong privacy and security
+- Anonymous public distributed ledger
+    - transactions are transparent, yet anonymous
+    - identity and personal activity is not a product
+- Each participant independently self-censors and decides acceptable speech
+    - participants can independently validate claims
+    - network facilitates claims management
+    - network facilitates dispute resolution

docs/img/dms3-1.jpg

+/home/tavit/Work/specs/dms3-1.jpg
\ No newline at end of file

docs/index.md

+
+## Home
+
+Welcome to the DMS3 documentation site.
+
+For more information on DMS3, please visit [dms3.io](https://dms3.io).

docs/intro.md

+
+## Introduction to DMS3
+
+DMS3 stands for **Dis-intermediated Multi-State Space Search**.
+
+DMS3 is a p2p network protocol that enables ***Search*** for
+information published by peer nodes.
+
+Peer nodes use DMS3 to index information into local repositories that enable context specific search for various kinds of information. For each indexed information repository, peer nodes also index metadata repositories used to provide context on locally hosted information. Metadata repositories also act as source testament to the nature of information hosted.
+
+Metadata repositories enable a search client to self-sensor inappropriate
+information while allowing the publisher freedom of speech. A metadata
+repository acts as a manifest for hosted information repositories.
+
+Metadata repositories enable network users to flag inconsistencies between
+published information and its manifest as an aid to expose deceptive peer
+nodes practices. Deceptive peer nodes can be quarantined and penalized by
+network protocols subject to arbitration resolution protocols.
+
+***Multi-State Space Search*** refers to the ability to separately search
+context specific manifest and information repositories.
+
+Information flow is ***Dis-intermediated***. Information is neither aggregated
+by a central service, nor filtered or rank ordered by a central service.
+
+### Key tenets
+
+#### Decentralized control
+
+DMS3 is a permission-less peer-to-peer (p2p) network of nodes that join and
+leave the network at will without a central authority enforcing admission control.
+
+#### Dis-intermediated service
+
+DMS3 services are transacted directly between participant nodes without third
+party mediation.
+
+#### Transparency of application and information
+
+Published DMS3 integrated applications are audited for transparency and security, such applications use the DMS3 API to store and publish generated persistent information.
+
+Applications run on endpoint nodes and store all local activity data on the
+local endpoint node via the DMS3 API.
+
+There are no web servers to collect any personal activity data using the application.
+
+All data sharing is controlled by the authorized endpoint node user. The
+network may offer optional centralized services for data availability, reliability, safety, and recovery. Additional future, new specialized network services may also be provided where implementing such services at each endpoint node is not capital efficient.
+
+#### Sovereignty and privacy of personal information
+
+The endpoint node user has full control over information sharing. Publishing information is explicitly controlled at each endpoint node by opt-in mechanisms. All information published by the endpoint node user is subject to transparency.
+
+Endpoint identity (ex: wallet account, node id, etc.) is transparent and opaque using public key cryptography mechanisms. Responsibility for  securing published information and personal endpoint accounts rests with  the endpoint user.
+
+#### Security in an untrusted environment
+
+Endpoint nodes in the p2p network are not trusted, trust is provided by the network layer using blockchain technology.
+
+#### Information availability and reliability
+
+The network implements optional centralized, permission-ed, shared nothing distributed architecture to provide information distribution and recovery services to protect against loss or availability of information.
+
+The p2p network uses *Smart Contracts* to enforce distribution and recovery
+service agreements between an endpoint and network services.
+
+#### Censorship, freedom, and acceptable speech
+
+Decentralization and Dis-intermediation enables freedom of speech.
+
+The network enables consumers to self-sensor inappropriate speech while
+enabling publishers the freedom of speech.
+
+Acceptable speech is defined by each endpoint and in accordance to locally
+interpreted cultural norms.
+
+The network protocols provide mechanisms to quarantine and penalize peer nodes that violate the acceptable use policy.

docs/js/converter.js

+
+(function () {
+'use strict';
+
+/**
+ * Targets special code or div blocks and converts them to UML.
+ * @param {object} converter is the object that transforms the text to UML.
+ * @param {string} className is the name of the class to target.
+ * @param {object} settings is the settings for converter.
+ * @return {void}
+ */
+var uml = (function (converter, className, settings) {
+
+  var getFromCode = function getFromCode(parent) {
+    // Handles <pre><code>
+    var text = "";
+    for (var j = 0; j < parent.childNodes.length; j++) {
+      var subEl = parent.childNodes[j];
+      if (subEl.tagName.toLowerCase() === "code") {
+        for (var k = 0; k < subEl.childNodes.length; k++) {
+          var child = subEl.childNodes[k];
+          var whitespace = /^\s*$/;
+          if (child.nodeName === "#text" && !whitespace.test(child.nodeValue)) {
+            text = child.nodeValue;
+            break;
+          }
+        }
+      }
+    }
+    return text;
+  };
+
+  var getFromDiv = function getFromDiv(parent) {
+    // Handles <div>
+    return parent.textContent || parent.innerText;
+  };
+
+  // Change body to whatever element your main Markdown content lives.
+  var body = document.querySelectorAll("body");
+  var blocks = document.querySelectorAll("pre." + className + ",div." + className
+
+  // Is there a settings object?
+  );var config = settings === void 0 ? {} : settings;
+
+  // Find the UML source element and get the text
+  for (var i = 0; i < blocks.length; i++) {
+    var parentEl = blocks[i];
+    var el = document.createElement("div");
+    el.className = className;
+    el.style.visibility = "hidden";
+    el.style.position = "absolute";
+
+    var text = parentEl.tagName.toLowerCase() === "pre" ? getFromCode(parentEl) : getFromDiv(parentEl);
+
+    // Insert our new div at the end of our content to get general
+    // typeset and page sizes as our parent might be `display:none`
+    // keeping us from getting the right sizes for our SVG.
+    // Our new div will be hidden via "visibility" and take no space
+    // via `position: absolute`. When we are all done, use the
+    // original node as a reference to insert our SVG back
+    // into the proper place, and then make our SVG visible again.
+    // Lastly, clean up the old node.
+    body[0].appendChild(el);
+    var diagram = converter.parse(text);
+    diagram.drawSVG(el, config);
+    el.style.visibility = "visible";
+    el.style.position = "static";
+    parentEl.parentNode.insertBefore(el, parentEl);
+    parentEl.parentNode.removeChild(parentEl);
+  }
+});
+
+(function () {
+  var onReady = function onReady(fn) {
+    if (document.addEventListener) {
+      document.addEventListener("DOMContentLoaded", fn);
+    } else {
+      document.attachEvent("onreadystatechange", function () {
+        if (document.readyState === "interactive") {
+          fn();
+        }
+      });
+    }
+  };
+
+  onReady(function () {
+    if (typeof flowchart !== "undefined") {
+      uml(flowchart, "uml-flowchart");
+    }
+
+    if (typeof Diagram !== "undefined") {
+      uml(Diagram, "uml-sequence-diagram", { theme: "simple" });
+    }
+  });
+})();
+
+}());

docs/js/main.js

+/** js sequence diagrams 1.0.6
+ *  http://bramp.github.io/js-sequence-diagrams/
+ *  (c) 2012-2015 Andrew Brampton (bramp.net)
+ *  @license Simplified BSD license.
+ */
+(function () {
+	"use strict";
+	/*global Diagram */
+
+	/*> ../build/diagram-grammar.js */
+	/*> ../fonts/daniel/daniel_700.font.js */
+	/*> sequence-diagram.js */
+	/*> jquery-plugin.js */
+
+	// Taken from what underscore.js:
+	// Establish the root object, `window` (`self`) in the browser, or `global` on the server.
+	// We use `self` instead of `window` for `WebWorker` support.
+	var root = (typeof self == 'object' && self.self == self && self) ||
+		(typeof global == 'object' && global.global == global && global);
+
+	// Export the Diagram object for **Node.js**, with
+	// backwards-compatibility for their old module API. If we're in
+	// the browser, add `Diagram` as a global object.
+	if (typeof exports !== 'undefined') {
+		if (typeof module !== 'undefined' && module.exports) {
+			exports = module.exports = Diagram;
+		}
+		exports.Diagram = Diagram;
+	} else {
+		root.Diagram = Diagram;
+	}
+
+}());

docs/what.md

+
+## What is the DMS3 Mission?
+
+- Dis-intermediated dissemination of information
+    - Allow participants to self-publish and share information into the peer-to-peer network
+        - Add information into DMS3 local endpoint repositories for indexing. Information may consist of:
+            - documents, tweets, messages,
+            - blogs, podcasts, media,
+            - news, research, web apps, and
+            - other articles, etc.
+        - Publish information in local repositories to allow others on the network to also search and access the information you want to share
+
+
+- Dis-intermediated retrieval of information
+    - Allow participants to find information published into the peer-to-peer network
+        - Search information in local repositories
+        - Search information published by others into the network
+
+    - Allow participants to opt-in to new information alerts
+        - kind of information to follow
+        - peer sources of interest to follow
+
+    - Allow participants to filter out unacceptable forms of information
+        - content matching filters, templates, and policies
+        - content source filters, templates, and policies
+
+
+- Dis-intermediated monetization decisions for the information economy
+    - Pricing decision is made in isolation, independent of who the purchaser may be
+    - Purchasing decision is made in isolation, independent of who the seller may be
+
+
+- [Dis-]intermediated network service
+    - *No web server involved*
+    - *Cryptocurrency token* operated network
+    - smart contract enforced content distribution agreements
+    - Many services are Dis-intermediated
+    - Lower barrier, Fair, and competitive markets for intermediated services
+
+
+- Security in an untrusted network environment
+
+    - Censorship free publishing
+    - Self imposed retrieval censorship based on local norms
+    - Community based rating of information sources
+        - Transparent information source ranking rules
+        - Ex: does published manifest or metadata accurately describe data?
+        - Dispute resolution inter-mediation process
+
+    - A peer-to-peer network architecture
+    - Blockchain technology
+    - Smart contracts
+    - Cryptocurrency economics
+
+
+- Commitment to sharing innovation in technology
+    - Inspired by IPFS
+    - Inspired by Ethereum
+    - Open source technology
+    - Transparent technology
+    - Audited security

docs/why.md

+
+## Why do we need DMS3?
+
+The Web promised to enable easy sharing of content allowing access to everyone connected to the Internet.
+
+The advent of advertising supported business models helped commercial interest compete with incumbents using Web service technology and eventually displace the then predominantly shrink-wrapped software business models of incumbents.
+
+With growth and popularity of the Web, commercial interest evolved the Web into islands of information with walled access in order to protect and grow market share of the intermediary commercial interest, leading to a pay-to-play model of Web service delivery.
+
+Advertising supported business models divide user communities into consumer and business classes that have different and often conflicting interests. Business class users subsidize the service that consumer users access for free. Consumer users incur an indirect costs of personal privacy, and in many cases freely provide content that fuels the service with little personal benefit and no leverage.
+
+In this model, a business or anyone really, pays to influence consumers with messages of own choosing. The intermediary has incentive to derive revenue enabling senders to influence target receivers of the service. This intermediary controlled push-based information delivery model is rife with risks.
+
+- Influence peddling
+- Bullying
+- Fake news
+- Unacceptable speech or inappropriate content
+- Unsolicited or sponsored messaging
+- Information bias or influence
+- Censorship
+- Privacy violation
+- Impersonation or identity theft
+- Targeted security threats (ex: ransomeware)
+- General security threats (ex: Network and Cloud infrastructure including DNS, Cert, and Web servers,)
+- etc...
+
+It is difficult to for the intermediary to address these challenges in the current Web service delivery model, and yet there are no alternatives to the consumer that enables service access while avoiding many of the above pitfalls.
+
+Information flow drives economic wealth and power. In the current push-based flow model few large firms act as gatekeepers to information flow.
+
+### Our Vision
+
+Our **_vision_** is to offer a pull-based information flow model as an alternative network that can avoid third party intermediary gatekeepers for most information services that are appropriate to run on commodity endpoint nodes, and enable broad competition for intermediary services that require specialized knowledge or resources to realize by most consumers.
+
+#### Dis-Intermediation of Information Flow
+
+Dis-Intermediation of Information Flow means removing the middleman that acts as a gatekeeper of information flow.
+
+Information flow is bidirectional. A source endpoint may **_disseminate_** information to a network service as in the case of posting to a social network. A source endpoint may **_retrieve_** information from a network service as in the case of using a search service for information discovery.
+
+##### Dissemination
+
+Centralized Information Feeds are mediated by a central authority.
+
+We all rely on the benefits of social networks for sharing information with personal and professional contacts, however **the centralized nature of information feeds coupled the human tendency to trust published information has generated new concerns** [Citations](#citations).
+
+- Pay-to-play
+- Sponsored information
+- Monopolistic competition
+- Journalistic objectivity
+- Fake news
+- Influence peddling
+- Censorship
+
+##### Retrieval
+
+Centralized Search Engine Ranking is mediated by a central authority
+
+We all rely on the benefits of quick retrieval capabilities offered by search engines, however **the centralized nature of result ranking coupled with the human tendency of short attention spans for reference searching has generated new concerns** [Citations](#citations).
+
+- Pay-to-play
+- Sponsored information
+- Monopolistic competition
+- Censorship
+- Rank gaming
+- SEO industry
+
+#### Censorship, Freedom, and Acceptable Content
+
+##### Censorship and Freedom of Speech
+
+Uninhibited advances in technology brings many benefits and is driving innovation at an accelerated pace. Regulatory systems attempt to attenuate the negative consequences of the  uninhibited flow of information whether for the enforcement of censorship or for the freedom of speech. For many reasons, regulatory systems move slower than innovation, and often react to catch up with the accelerated pace of technological innovation. **By the time new regulations take effect, innovators have moved to newer _solution_ grounds** [Citations](#citations).
+
+- Censored search
+- Insider trading
+- Journalistic objectivity
+- Fake news
+- Influence peddling
+- Politics of Identity
+- Truths and Falsehoods
+
+
+##### Acceptable Norms of Content and Speech
+
+How to isolate acceptable speech from unacceptable speech is a difficult problem seeking better solutions. Current approaches that apply human capital and artificial intelligence technology do not offer a silver bullet. Acceptable norms differ in various locales and segments of society, reliance on **a centralized approach to determine acceptable speech and filter unacceptable speech poses other challenges** [Citations](#citations).
+
+- Increased cost and economic drag
+- Censorship
+- Discrimination
+- Biased perspectives
+- Defamation of character
+- Bullying
+
+#### Information Economy and The Economic Gap
+
+In an information economy, controlling access-to and flow-of information drives economic value. Centralized control over information retrieval and dissemination tends to concentrate power, offers asymmetric leverage that drives growth, and encourages monopolistic competition. **Centralization of power, growth, and competition poses a number of challenges** [Citations](#citations).
+
+- Creates drag on productivity and economic growth
+    - Leads to less competitive markets
+    - Raises barriers to entry
+    - Encourages monopolistic competition
+- Concentrates economic risk
+    - Individual risk
+        - Data loss, privacy, misuse, theft, censorship, trust requirement
+    - Network risk
+        - Too Big to Fail
+    - Geopolitical risk
+        - Migrate operations to geographies offering favorable regulations and costs
+        - Opaque supply chain and the misaligned interests of producers and consumers
+            - Health, environment, and ecosystem risks shift to less regulated geographies
+            - Capital risk is exposed to foreign relations, trade politics, and corruption
+- Widens The Economic Gap
+    - Narrower set of beneficiaries of technology and economic gains
+    - Acts as drag on wage growth
+        - Large global corporations benefit from capital investment migration
+        - Wage earner migration has many impediments, is slow, and poses social and economic costs at best
+
+#### Citations
+
+##### [Competitive markets](http://www.economicsonline.co.uk/Competitive_markets/Competitive_markets.html)
+
+"A competitive market is one in which a large numbers of producers compete with each other to satisfy the wants and needs of a large number of consumers. In a competitive market no single producer, or group of producers, and no single consumer, or group of consumers, can dictate how the market operates. Nor can they individually determine the price of goods and services, and how much will be exchanged. Competitive markets will form under certain conditions."
+
+##### [The tech sector is leaving the rest of the US economy in its dust Apple, Alphabet, Facebook, Amazon, and Microsoft By Ben Popper May 16, 2017, 9:18am EDT](https://www.theverge.com/2017/5/16/15627198/tech-sector-stock-market-record-high)
+
+"As these companies continue to scale, the network effects bolstering their business are strengthening. Facebook and Google accounted for over three-quarters of the growth in the digital advertising industry in 2016, leaving the rest to be divided among small fry like Twitter, Snapchat, and the entire American media industry. Meanwhile Apple and Alphabet have achieved a virtual duopoly on mobile operating systems, with only a tiny sliver of consumers choosing an alternative for their smartphones and tablets."
+
+"In another era, Google dominance in search, Facebook’s commanding lead in social, or Amazon’s massive reach in e-commerce might lead regulators to view them as a monopoly. There has been some pushback in Europe, but so far US watchdogs haven’t taken any aggressive action."
+
+"Writing in the Yale Law Journal earlier this year, Lina Kahn argued that “the current framework in antitrust—specifically its pegging competition to ‘consumer welfare,’ defined as short-term price effects—is unequipped to capture the architecture of market power in the modern economy.”"
+
+"By positioning themselves as the platforms on top of which information technology across every sector now runs, these titans of tech become the new centers of gravity for our economy, growing in size, scope, and influence while everyone else struggles not to fall too far behind."
+
+
+##### [Tech is (still) concentrating in the Bay Area: An update on America’s winner-take-most economic phenomenon Mark Muro and Jacob Whiton Monday, December 17, 2018](https://www.brookings.edu/blog/the-avenue/2018/12/17/tech-is-still-concentrating-in-the-bay-area-an-update-on-americas-winner-take-most-economic-phenomenon/)
+
+"While employment in tech is growing all over America, it really isn’t “spreading out” in terms of more cities gained increased shares of the tech pie."
+
+"Even while tech continues to raise hopes for broad transformation, it is continuing to reflect—and drive—the winner-take-most nature of the American economy."
+
+##### [As Facebook Raised a Privacy Wall, It Carved an Opening for Tech Giants](https://www.nytimes.com/2018/12/18/technology/facebook-privacy.html)
+
+"By Gabriel J.X. Dance, Michael LaForgia and Nicholas Confessore
+Dec. 18, 2018"
+
+“This is just giving third parties permission to harvest data without you being informed of it or giving consent to it,” said David Vladeck, who formerly ran the F.T.C.’s consumer protection bureau. “I don’t understand how this unconsented-to data harvesting can at all be justified under the consent decree.”
+
+##### [Facebook’s Data Sharing and Privacy Rules: 5 Takeaways From Our Investigation](https://www.nytimes.com/2018/12/18/us/politics/facebook-data-sharing-deals.html)
+
+"By Nicholas Confessore, Michael LaForgia and Gabriel J.X. Dance
+Dec. 18, 2018"
+
+"** _You are the product_ **: That is the deal many Silicon Valley companies offer to consumers. The users get free search engines, social media accounts and smartphone apps, and the companies use the personal data they collect — your searches, “likes,” phone numbers and friends — to target and sell advertising."
+
+##### [Google isn’t the company that we should have handed the Web over to](https://arstechnica.com/gadgets/2018/12/the-web-now-belongs-to-google-and-that-should-worry-us-all/)
+
+"Analysis: Microsoft adopting Chromium puts the Web in a perilous place.
+Peter Bright - 12/17/2018, 5:19 PM
+"
+
+##### [Some of YouTube's biggest stars say their work is hurting their health](https://www.marketplace.org/2018/11/28/tech/some-youtubes-biggest-stars-say-youtube-bad-their-health)
+
+"If you are a creator on YouTube, you live and die by the algorithm. It's the invisible technology that recommends videos on the trending page or in search results or suggestions all across the platform. No one but YouTube knows exactly how the algorithm decides what to promote and what not to promote. The controversial process has at times promoted fake news and sensational or harmful content. And creators say it's burning them out."
+
+##### [Google employees demand that Google stop work on censored Chinese search](https://arstechnica.com/tech-policy/2018/11/google-employees-demand-that-google-stop-work-on-censored-chinese-search/)
+
+"Dozens of Google employees have signed on to an open letter demanding that Google stop work on Project Dragonfly, a censored version of Google's search engine that could be deployed in mainland China."
+
+##### [Study: Over 20 years, Silicon Valley workers’ median wage has fallen by 14%](https://arstechnica.com/tech-policy/2018/11/study-over-20-years-silicon-valley-workers-median-wage-has-fallen-by-14/)
+
+Prof: "The returns to capital are significantly outpacing the returns to labor."
+
+"Over the last two decades, 90 percent of workers in San Mateo and Santa Clara counties—the heart of Silicon Valley—have seen their real wages go down, according to a new study by the University of California, Santa Cruz and the think tank Working Partnership USA."
+
+"In short, most workers—regardless of whether they work in the tech sector or not—are getting poorer due to venture capital-driven business models that prioritize outlandish returns fueled by low-wage work that captures a given market quickly."
+
+##### [When a network intel provider’s domain serves fraudulent content, something is wrong](https://arstechnica.com/information-technology/2018/11/network-intel-providers-domain-served-fraudulent-content-for-2-weeks/)
+
+"By being available on a subdomain of a legitimate network intelligence company, the content was designed to manipulate Google search results in a way that tricked people into clicking on questionable links."
+
+##### [The Love of One's Own and the Importance of Place](https://worldview.stratfor.com/article/love-ones-own-and-importance-place)
+
+"The study of geopolitics tries to identify those things that are eternal, those things that are of long duration and those things that are transitory. It does this through the prism of geography and power. More precisely, geopolitical inquiry seeks not only to describe but also to predict what will happen. Those predictions frequently — indeed, usually — fly in the face of common sense. Geopolitics is the next generation’s common sense."
+
+"The idea that individual and choice supersede community and obligation is embedded in the American Declaration of Independence, which elevates life, liberty and the pursuit of happiness over obligation."
+
+"Place defines enemies, fears, actions and, above all, limits."
+
+"Societies and people run on different clocks. This is the fundamental tension between a nation and an individual."

mkdocs.yml

+
+site_name: DMS3 documentation
+
+nav:
+  - Home: index.md
+  - Why do we need DMS3?: why.md
+  - What is the DMS3 Mission?: what.md
+  - How is DMS3 Better?: how.md
+  - Introduction to DMS3: intro.md
+  - Architecture Overview: arch.md
+  - About: about.md
+###  - Example Math and Diagrams: diagrams.md
+
+theme:
+#  name: readthedocs
+  name: rtd-dropdown
+  custom_dir: docs/
+
+markdown_extensions:
+  - footnotes
+  - pymdownx.arithmatex
+  - pymdownx.superfences:
+      custom_fences:
+        - 'name': 'flow'
+          'class': 'uml-flowchart'
+          'format': !!python/name:pymdownx.superfences.fence_code_format
+        - 'name': 'sequence'
+          'class': 'uml-sequence-diagram'
+          'format': !!python/name:pymdownx.superfences.fence_code_format
+        - name: 'mermaid'
+          class: 'mermaid'
+          format: !!python/name:pymdownx.superfences.fence_div_format
+        - name: 'math'
+          class: 'arithmatex'
+          format: !!python/name:pymdownx.arithmatex.fence_mathjax_format
+
+extra_css:
+#  - https://unpkg.com/mermaid@7.1.2/dist/mermaid.css
+extra_javascript:
+  - 'js/converter.js'
+  - 'js/mermaid.min.js'
+  - 'js/mermaid.min.js.map'
+  - 'https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.9.1/underscore-min.js'
+  - 'https://cdnjs.cloudflare.com/ajax/libs/underscore.js/1.9.1/underscore-min.js.map'
+  - 'https://cdnjs.cloudflare.com/ajax/libs/raphael/2.3.0/raphael.min.js'
+  - 'https://cdnjs.cloudflare.com/ajax/libs/flowchart/1.11.3/flowchart.min.js'
+  - 'https://cdnjs.cloudflare.com/ajax/libs/flowchart/1.11.3/flowchart.min.js.map'
+  - 'https://cdnjs.cloudflare.com/ajax/libs/js-sequence-diagrams/1.0.6/sequence-diagram-min.js'
+  - 'https://cdnjs.cloudflare.com/ajax/libs/js-sequence-diagrams/1.0.6/sequence-diagram-min.js.map'
+  - 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.6/MathJax.js?config=TeX-MML-AM_CHTML'

wip/diagrams.md

+
+## Example Math and Diagrams
+
+```math
+\begin{align}
+    p(v_i=1|\mathbf{h}) & = \sigma\left(\sum_j w_{ij}h_j + b_i\right) \\
+    p(h_j=1|\mathbf{v}) & = \sigma\left(\sum_i w_{ij}v_i + c_j\right)
+\end{align}
+```
+
+\[3 < 4\]
+
+$$
+\frac{n!}{k!(n-k)!} = \binom{n}{k}
+$$
+
+Lorem ipsum dolor sit amet: $p(x|y) = \frac{p(y|x)p(x)}{p(y)}$
+
+
+```sequence
+    Alice ->> Bob: Hello Bob, how are you?
+    Bob-->>John: How about you John?
+    Bob--x Alice: I am good thanks!
+    Bob-x John: I am good thanks!
+    Note right of John: Bob thinks a long<br/>long time, so long<br/>that the text does<br/>not fit on a row.
+
+    Bob-->Alice: Checking with John...
+    Alice->John: Yes... John, how are you?
+```
+
+```sequence
+Title: Here is a title
+A->B: Normal line
+B-->C: Dashed line
+C->>D: Open arrow
+D-->>A: Dashed open arrow
+```
+
+```flow
+st=>start: Start:>http://www.google.com[blank]
+e=>end:>http://www.google.com
+op1=>operation: My Operation
+sub1=>subroutine: My Subroutine
+cond=>condition: Yes
+or No?:>http://www.google.com
+io=>inputoutput: catch something...
+
+st->op1->cond
+cond(yes)->io->e
+cond(no)->sub1(right)->op1
+```
+
+```mermaid
+graph LR;
+  A-->B;
+  B-->C;
+  C-->A;
+```
+
+```mermaid
+graph TD;
+  A-->B;
+  A-->C;
+  B-->D;
+  C-->D;
+```