The place for everything full stack. - Medium

Everything Full Stack - Medium

<h3>Do you have the Basics of Functional Tests?</h3>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RnoX1gFJSfGLrjTS5M02fw.png"></figure>It is the most common test type. As such, almost every coder learned it by example, alas — very few understand it in depth.
The internet is so full of code-snippets, that too few take the effort to understand above the minimum it takes to make them work in a project. This created a very low bar: So much so that even good and talented authors of recommended test tools provide them with examples that demonstrate how their tool is used, but mislead on how to write a good test.
<blockquote>This post goes through the core concepts of Functional Tests and brigs up methods to increase their maintainability.</blockquote>
<blockquote>If you’re interested in the ecosystem they’re a part of — you may read on other test types and why to perform each of them on the previous post about <a href="https://medium.com/everything-full-stack/do-you-waterproof-your-software-e9ba250bd8bd">proofing software in CI</a>. To understand CI/CD — read the post about <a href="https://medium.com/everything-full-stack/whats-exactly-ci-cd-8e943902f38e">what is CI/CD</a>.</blockquote>
<h3>First — a term: SUT = System Under Test</h3>
SUT — stands for System-Under-Test, which is a generalization for Unit-Under-Test, Class-Under-Test, Endpoint-Under-Test, Component-Under-Test, Service-Under-Test, Software-Under-Test etc. — all systems. So yeah. SUT.
<h3>What are Functional Tests?</h3>
Functional tests check if a working SUT provides the desired result expected of it.
Consider a mathematical approach: any SUT works in a context of a set of parameters, and yields a result — like a mathematical function — hence the name. That’s all.
<h4>Not related to functional code!!</h4>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oIJTHtcVSBT_x0oZp4KYXQ.png"><figcaption>not our discussion</figcaption></figure>The discussion about functional code and distinctions about the returned value and side-effects — is an important and tempting discussion, but this is not our current discussion!
<blockquote>On the contrary: the goal is a GENERIC definition that will suit ANY code whatsoever, functional or not.</blockquote>
<h4>What Functional Tests AREN’T (or, what test runners ARE)?</h4>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Un9HEwRJtL6Pz67WLC7ghA.png"></figure>Purely speaking, Functional Tests are not interested in how much time the SUT took to get to the expected result. Or how much resources it occupied to do so, or if it did its job efficiently. Only that the result is correct. That’s purely speaking.
In practice — most test tools come with inherent features outside the domain of functional testing — e.g. timeout mechanisms: but when you ask if it did it in time — you ask how long it took, and that’s a performance measure.
<blockquote>The takeaway is that the type of test is a logical thing and is not an implication of the tool you’re using.</blockquote>
<blockquote>On the contrary- Good test tools aspire to let you do everything, if only they could.</blockquote>
Everything? Yes. Functional, performance, coverage, load, bench — anything that runs.
That’s why test runners are in their core task managers: Your test-cases tree is your task list. You could run them all (like in CI) — but you can run selective tasks if you must (like in debugging a single test-case).
<blockquote>This has an implication: ideally, test cases must not depend on each other. Each test is a task which should be runnable on its own.</blockquote>
When a case depends on other cases being run before — it’s called case sliding. The upside — in very heavy systems, while you’re at it — you just might check one more pesky thing… The downside — it doesn’t give a reliable status. When a case fails — all the cases that come after it probably won’t even run, or will run and give misleading failures.
<h3>func(params) = expected</h3>
No, that’s not an assignment. That’s a mathematical statement. It evaluates to true or false, which correlates to PASS or FAIL.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/915/1*joKP2ZtYh5c22sCgIdH3zw.png"><figcaption>accurate correct result</figcaption></figure><h4>The expected result</h4>
To be able to tell if a function worked as expected — we need to know what result we’re expecting.
A common misbelief about the expected result is that it is a single value. IMHO, this is a legacy of the early programming languages which inspired each other, that tried to define functions by the type of their returned value, and thus, require their functions to return a single value. Anything outside this returned value ended up being called a side-effect, which is a cheap cheat and a loop-hole in terms of testing, designed to allow us to ignore things the function does, even though it’s definitely being done by our code — and therefore must be accounted for and tested.
Another common workaround is that a single returned value may be a data-structure. But then, you still need to verify every part of that structure you’re interested in, thus — a data structure is a Set of results, just like the set of the returned value and any other values assigned to global, delivered by reference, or passed as a message by the SUT.
<h4>The business-level calls them objectives</h4>
Now that we have acknowledged that a result can consist of several values, we’ll go up to the business level: the expected result often includes a few objectives.
Consider the following example: A successful registration means:
<ol>
<li>a new user is created — with the correct fields
</li>
<li>an email has been sent — with the expected message
</li>
<li>a log line has been produced — with the expected information
</li>
<li>confirmation message has been displayed— with correct data
</li>
</ol>
<blockquote>The key takeaway is that ANY effect that is caused by the SUT may be a part of the expected result.</blockquote>
<blockquote>BTW — effects that are applied and aren’t required are likely to be a waste!
</blockquote>
<h4>The causality of parameters</h4>
Causality — describes a relation between cause and effect. Purely speaking, parameters propagate through the SUT and together they cause a result.
Parameters — in their wider sense: can be an input, an inner state, or an outer state. E.g.:
<ul>
<li>
add(a: Number, b: Number): Number — takes input
</li>
<li>
count(): Number — relies on inner state</li>
<li>
fetchNext(): Entity— talks to an external system</li>
</ul>
Obviously, SUT can work with any combination of the three.
Some parameters are totally in our control, like a and b of the add. Some are not in our hands — like the current time, which is constantly moving, or an answer from an external system that is not in our control.
If we cannot set them ourselves — we should be able to sample them. When the sampling itself incurs a change of state — the test code must account for that too.
<blockquote>Identify and account for ALL the parameters affecting the expected result.</blockquote>
<h3>The AAA Method (Arrange, Act, Assert)</h3>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*C-mX2u7xR1541q3TuhHHEw.png"><figcaption>Powered by AAA</figcaption></figure>This is a theorem that comes to increase maintainability.
<blockquote>When something goes wrong in the test — how easily can you spot the culprit?</blockquote>
It directs code-organization around the following test-concerns:
<h4>Arrange</h4>
<ul>
<li>This organizes all the parts the SUT comes to contact with, finds their state or set it explicitly.</li>
<li>As such, it includes constructors and factory calls, preparing mocks, laying traps, and/or IO operations to remote systems.</li>
<li>We may not succeed to do just that — and the test-case will fail.</li>
</ul>
A failure here means the test code and/or any of the helpers it uses are broken: The test-code did not even get to interact with the SUT.
<h4>Act</h4>
<ul>
<li>This is the part where we interact with the SUT to operate it and gather all the results.</li>
<li>This will be a single API call: a method call, a fire of an event, a send of a message, or even a network request. In some cases, there will be more data-gathering IO operations right after.</li>
<li>We may not succeed in performing the action at all — and the test will fail.</li>
</ul>
A failure here will mostly mean that the SUT is not resilient or is unstable. When the test-code does not communicate well with the SUT — we expect the SUT to communicate it well in the errors it throws.
<h4>Assert</h4>
<ul>
<li>This is the part where we go over every value in the gathered results, and make sure that it is what it is expected to be. All values have been gathered — now we validate them.</li>
<li>It runs completely synchronously, entirely in memory.</li>
<li>This part is the Purpose of the test: This is where we validate that the SUT is doing what it’s expected to do — that everything that should happen did, and that no undesired result is left in the world the SUT lives in.</li>
<li>
Ideally, each assertion will produce in the report a single title. Obviously, there are shortcuts — like assertions that compare an entire data-structure.</li>
</ul>
A failure here means the SUT does not meet its spec.
The AAA is something that is found on the web in many sources. Let’s add to that a small contribution:
<figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*gNc1xH2E84QjAo1VJtG_GQ.png"><figcaption>AAAA — lighter, smarter, cleaner and now — more powerful!</figcaption></figure><h4>Aftermath</h4>
<blockquote>Oh, I did not invent this practice — Arrange + Act together are known in textbooks as setup, and Aftermath is known as teardown or cleanup. I just chose a name that starts with A to go with the other AAAs</blockquote>
<ul>
<li>This is where we make sure that the test suite itself did not leave any undesired results on the world it lives in.</li>
<li>It’s made of untie mocks and traps, destructors, memory deallocations, and/or IO operations to remote systems.</li>
</ul>
A failure here is a bug in the Test-Code.
<blockquote>This stage sometimes seems to be neglected, especially when the test-suite runs in complete isolation with databases that are spun-up and destroyed at the end of the entire suite. This does not mean this concern is not there — it’s just moved from the test-cases code to the orchestration level. I mean to get into how to leverage that in a future post.</blockquote>
<blockquote>The key takeaway is the distinction between the stages. When a test fails — the stage that fails tells us what should be worked out.</blockquote>
<blockquote>We communicate it to future maintainers by organizing code in the 4 test concerns and externalize it all the way to the the test report.</blockquote>
<h3>The Trinity of Tests</h3>
This trinity does not deal with divinity, but makes a test suite self-explanatory to increase its maintainability.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RBF3jc0foJwiV9yrPPDslg.png"></figure><h4>The S.U.T.</h4>
Often it’s a part of production-code. But it can be a test-helper or the test itself.
<blockquote>
Here’s a mind-bender: when you measure coverage, they change roles: your SUT is your tests suite — you want to know if it covers you well, and the production code gets to be in the role of the Test-Code...</blockquote>
<h4>The Test Code</h4>
This is the element that interacts with the SUT and measures its effects. This includes any test-helpers, test data-fixtures, and the implementation of the test-cases with their setup/teardown and their assertions.
<h4>The Spec</h4>
This part was suppressed from tests for too long. Systems like JUnit and the rest of the (X)Unit family originally let coders snuck the specs into names of test-methods and classes — which was very limiting, and the rest was either truncated and lost or in the better cases snuck into comments and was not apparent in reports. Although we have evolved since, the entire industry still bears the scars of this past.
<h4>Ok, nice. How is this useful?</h4>
Often tests are very clear once written, but become gotcha-full head-scratchers even for their author when she meets them in the future.
<blockquote>1 — It’s all about what the test communicates to its readers</blockquote>
When a coder comes across a failing test — she needs to take judgment. When the intent of the test is not clear, the coder has to speculate too much in order to judge between the test and the SUT:
<ul>
<li>In the midst of bug hunt: is the bug in the SUT or in the Test-Code?</li>
<li>Is it a matter of evolution? Did the SUT evolve and the test did not, and now the test expresses a requirement that is no longer relevant? Or does the SUT now break a spec?</li>
</ul>
<blockquote>2 — The thing is: Trinities do not divide in two</blockquote>
The spec helps to judge between the SUT and the Test-Code. Usually the spec agrees with one, indicating that the other should adhere. But I’ve seen cases where the SUT and the Test-Code agree, and the title is wrong (a poor copy-paste).
<h3>Putting it all together</h3>
The bad example — in the following <a href="https://gist.github.com/osher/51237ee8f74a7c18852799de9393bd48">gist</a>:
<a href="https://medium.com/media/9d1d0338e0ff2c2aaade56f21631212d/href">https://medium.com/media/9d1d0338e0ff2c2aaade56f21631212d/href</a>The better example — in the following <a href="https://gist.github.com/osher/4a08b0c633e41919298f0f6ce9357b92">gist</a>:
<a href="https://medium.com/media/4063939e446c24061008bd339df42ab1/href">https://medium.com/media/4063939e446c24061008bd339df42ab1/href</a><h4>Oi, the verbosity</h4>
It is not much about the verbosity as it is about the intent to leave clues for all your future readers:
<ul>
<li>The reader of the report in the CI output</li>
<li>The reader of the test-report ran in the IDE</li>
<li>The reader of the coverage report</li>
<li>The reader of the code itself</li>
</ul>
Please note: the specs appear in all of them, while the code appears only in the last. Make sure your breadcrumbs get to ALL of them.
I confess that when the entire test-case is for a single assertion, even I am tempted to do the entire four A’s in a single it(..) . But I do make sure to leave enough info about what I mean the test to do as part of what context.
<blockquote>The takeaway is — whatever titles you write , make sure that the they explain your intent and are unambiguous…</blockquote>
<h4>Test Factories</h4>
This is worth a post of its own, but I will not come clean if I do not mention it.
Often a set of test-cases require the same setup and teardown. Given the recommended test verbosity demonstrated above — a naïve implementation will result in some copy-pastes with poor maintainability.
The Wise packs the repetitive code in test-helpers that assist in these repeating setup / teardown tasks. The Even Wiser — feature a helper API that manages a complete test-context with setup and teardown hooks, providing the test-code with a context ready for the Assert stage.
A test-helper becomes a test-factory when it features a single API that expects arguments that declare not only how to perform any required setup and teardown of the test context, but also what has to be asserted as part of what spec. Calling it generates a complete test-case that appears in the report with readable titles.
For a simplistic example — pretend the language did not provide you with indexOf and you had to implement it yourself. Now consider the following <a href="https://gist.github.com/osher/b13feb201f89decd39482f8943266796">gist</a>:
<a href="https://medium.com/media/db5a4b89b2cfc23cc4339b08691231a0/href">https://medium.com/media/db5a4b89b2cfc23cc4339b08691231a0/href</a><h4>Do not be afraid to Delete</h4>
Once you get proficient with producing tests, when spec change, it often becomes easier to rewrite new tests instead of refactoring existing ones.
Given that you kept your test-cases isolated (no case sliding), kept your AAA(A) organized, and placed reusable code in appropriate helpers, writing a case should not take much time — it is a matter of declaring values.
It is hard to delete a case when you like its aesthetics. But if it has lost relevance and/or does not add information or coverage — get rid of it!
<h3>Conclusion</h3>
The goal of a test-suite is to affirm that our software works as expected. However, beyond this desired affirmation, when it is denied — we want to know what went wrong and what must be fixed. The first level of defense is pointing out what went wrong by organizing our test code into recognizable segments, the goals of which are defined by the AAA(A) theorem. To crosshatch this, we bring the spec into the test and communicate the goals of each test-case in the suite, so that when they fail we know what the failure points at without the need to read the test. And when they are refactored — we know to preserve these goals and maintain coverage.
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=5f8c975dd50f" width="1" height="1" alt=""><hr>
<a href="https://medium.com/everything-full-stack/are-you-professional-with-functional-tests-5f8c975dd50f">Are you professional with Functional Tests?</a> was originally published in <a href="https://medium.com/everything-full-stack">Everything Full Stack</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Are you professional with Functional Tests?

<figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/0*HV8MFyrLFHX7evI1.png"></figure>By the end of this post, we’ll build a small <a href="https://fastapi.tiangolo.com/#:~:text=FastAPI%20is%20a%20modern%2C%20fast,the%20fastest%20Python%20frameworks%20available.">FastAPI</a> server, send it a request and receive a response, and add authentication/authorization around a route!
Why FastAPI?
I chose FastAPI for this demo because of its growing popularity over the past couple of years:
<ul>
<li>It’s the <a href="https://insights.stackoverflow.com/survey/2021#section-most-loved-dreaded-and-wanted-web-frameworks">third most loved</a> web framework in StackOverflow 2021 Developer Survey</li>
<li>It is used by large companies like <a href="https://eng.uber.com/ludwig-v0-2/">Uber</a> and <a href="https://netflixtechblog.com/introducing-dispatch-da4b8a2a8072">Netflix</a> to develop some of their applications</li>
<li>It fully supports async which makes it really fast!</li>
<li>It has a declarative routing &amp; validation</li>
<li>It has an automatic documentation generation (swagger)</li>
</ul>
If you’re not convinced by now, I recommend <a href="https://christophergs.com/python/2021/06/16/python-flask-fastapi/">this blog post</a> for a more thorough overview!
Important Note I: the commands in this post were written &amp; executed on MacOS (on zsh), and they will probably work on any Linux distro, but if you’re working on windows you’ll have to find the parallel commends, sorry for that…
Second, This post assumes that you already have python3 and pip installed on your machine, in case this assumption is wrong: you can find a <a href="https://phoenixnap.com/kb/install-pip-mac">good explanation here</a> on how to do it.
Let’s begin by creating a new project folder:
<pre>mkdir server cd server</pre>
Now let’s create and activate a <a href="https://docs.python.org/3/library/venv.html">virtual environment</a>, in case venv is not installed we’ll want to first run:
<pre>python3 -m pip install --upgrade pip python3 -m pip install --user virtualenv</pre>
Otherwise, move on to:
<pre>python3 -m venv ./venv source ./venv/bin/activate # whenever you want to exit venv type: "deactivate"</pre>
If you’re not familiar with the advantages of using virtualenv with Python I can’t stress enough how important it is to take the time and study it, but for now, I’ll quote <a href="https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#:~:text=virtualenv%20is%20used%20to%20manage,can%20install%20virtualenv%20using%20pip.">the docs</a>:
<blockquote>
<a href="https://packaging.python.org/en/latest/key_projects/#virtualenv">virtualenv</a> is used to manage Python packages for different projects. Using virtualenv allows you to avoid installing Python packages globally which could break system tools or other projects.</blockquote>
Now let’s install the two dependencies that we need: FastAPI and PyJWT, but before we do that let’s make sure that pip is up-to-date:
<pre>python3 -m pip install --upgrade pip pip3 install "fastapi[all]" pip3 install PyJWT</pre>
Important Note II: the three steps we did so far, are good for any python project we’ll want to create in the future:
<ul>
<li>create a folder and go inside it</li>
<li>create venv and activate it</li>
<li>install the dependencies</li>
</ul>
Now we’re ready for the code!
let’s create a main.py file using our favorite IDE (I’m using <a href="https://code.visualstudio.com/">VSCode</a> or <a href="https://www.jetbrains.com/pycharm/">pycharm</a> when I write python) and paste the following code snippet:
<a href="https://medium.com/media/dcaa9789928e08ee09f3f89a5653b95b/href">https://medium.com/media/dcaa9789928e08ee09f3f89a5653b95b/href</a>Now we can run the code with:
<pre>python3 -m uvicorn main:app --reload</pre>
To test our server, we can use the following curl:
<pre>curl --location --request POST '<a href="http://127.0.0.1/">http://127.0.0.1:8000'</a> \ --header 'Authorization: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJoZWxsbyI6IndvcmxkIn0.bqxXg9VwcbXKoiWtp-osd0WKPX307RjcN7EuXbdq-CE' \ --header 'Content-Type: application/json' \ --data-raw '{ "name": "Foo", "gender": "male", "age": 25.5, "checked": false }'</pre>
Pay special attention to class Person — this is the way FastAPI wants us to work with Objects: in this example, we’re receiving an object from the client via a POST HTTP request and this person object should align with the expected structure of the class, that is, if we will not send a mandatory attribute we’ll receive a 422 unprocessable entity HTTP error.
The second thing I’d like to delve into is the signature of the route: the decorator @app.post("/")is pretty self-explanatory: it routes the POST requests to/ on our localhost — to this function. The interesting part is the parameters received by the function:
<pre>1. person: Person 2. authorization: str = Header(None)</pre>
This signature mandates that the body of the request will contain a JSON object with the structure dictated by Person class and that the request will have an Authorization header. We can make a header optional by doing the following:
<pre>authorization: Optional[str] = Header(None)</pre>
but there’s no point in doing it here since we’ll use this header to verify that the user who sent the request is authorized to perform this action.
Important Note III: this is the place to stress that this simplified example should NOT be used in production “as is”: PyJWT supports different algorithms for signing and we should use a stronger algorithm (e.g. RSA) or at least stronger secrets. The secrets should NEVER be hardcoded but rather saved as environment variables and approached via a config module. If you’re interested to learn more about these options, PyJWT docs have some <a href="https://pyjwt.readthedocs.io/en/latest/usage.html">nice examples</a>!
Before we part ways I would like to encourage you to play and expand this code sample. For instance, we can add a DB connection, add a Users Table and add code to verify that the user who sent the request has permission to create/modify the object. Then we can also add a Persons Table which will save the related data, and also add routes to modify and delete an existing person. For extra credit, you can improve the implementation of JWT Token that was presented here and use “RS256” algorithm with private and public certificates or learn how to work with “bearer token” (a nice example can be found <a href="https://testdriven.io/blog/fastapi-jwt-auth/">here</a>).
Enjoy your dive-in &amp; Good luck!
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=f4c03d96a8ea" width="1" height="1" alt=""><hr>
<a href="https://medium.com/everything-full-stack/securing-a-fastapi-route-using-jwt-token-step-by-step-f4c03d96a8ea">Securing a FastAPI route using JWT token (step-by-step)</a> was originally published in <a href="https://medium.com/everything-full-stack">Everything Full Stack</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Securing a FastAPI route using JWT token (step-by-step)

Do you Waterproof your Software?

Terminal escape codes are awesome, here’s why

TL;DR Engineering skills in DevOps are a necessity for scaling up. It isn’t something you pick up as you go along something, it’s something you should plan as part of your Career Journey.
As the DevOps role evolves, alongside the demand for scale ⚖️, we are also required to sharpen our engineering capabilities.
For senior developers, the tricks of the trade are usually based on their clear view of design patterns enabling them to identify them wherever they go. But for `DevOpsers` the situation is not always that straightforward…
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VueCV2tD1Wq4tDEp.png"><figcaption>Career Journey</figcaption></figure><h3>From Racks 2 Stacks</h3>
My personal journey started in the 90s as an IT professional with 128KBps dial-up connections and the need for speed was in demand in order to ship adverts on time to local pre-press and ad agencies. ⏩ 15+ years Fast forward a common stack could probably deal with the same amount of traffic we dealt with in a month, … same goes for setting that rack up…
<h3>⚛ ️Automation, Engineering, and Cloud</h3>
The “times ten” we are all seeking for isn’t exponential … meaning it took time to build the best practices and adopt architectural styles to achieve the same amount of capacity in faster, in one-word Automation, in a few more the ability to automate &amp; engineer a system — a.k.a systems engineering.
All this Cloud-Native goodness has come with an abundance of standards &amp; patterns we learned to appreciate and adopt.
In the early stages Batch files &amp; Shell scripts were all you needed on the scale of a few physical servers … with 10’s -&gt; 1000’s these, methods don’t suffice.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*YyleptHEgvNfxLMQ.jpg"><figcaption>⬅️ | Shifting left</figcaption></figure><h3>12factor, Ruby on Rails, Frameworks &amp; Fireworks</h3>
The compute consumption model has shifted from physical to virtual and eventually to the cloud. Servers are just someone else’s hardware and many companies are consuming computing in the forms of IaaS, PaaS, SaaS, or FaaS.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lYV59bW07fQPIDJS.jpg"><figcaption>Modern compute consumption models</figcaption></figure>These Computing models require us to program our infrastructure and manage our distributed applications across multiple architectures and cloud providers …
12Factor app principles and Frameworks such as Ruby on Rails, Ruby, in general, have defined a common standard of modern distributed system architecture.
<h3>From Stateless config to Controller loops</h3>
As PaaS services such as Heroku and the 12factorApp principles helped define the standard of modern web services, it also defined a more distributed and flexible way of thinking — this approach helped influence projects like Kubernetes which is now dominating container scheduling decoupling it from the cloud provider (as much as possible).
<figure><img alt="" src="https://cdn-images-1.medium.com/max/873/0*041i7BcvtjRNxCKM"><figcaption>Distributed what of Thinking</figcaption></figure>With the maturity of container technology and cloud services offering to run containers in various ways we now find ourselves with complex deployment models and a lack of skill to engineer it …
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sKgN0xLvJ45lxZOQ.png"></figure>The old way of doing Configuration Management was to use an ad-hoc tool to do the job e.g. when a server launches, bootstrap it, configure it, test it, connect it… a repetitive task that for the Engineers was shortly an Ansible/Chef/Puppet module you’d distribute and use in your automation.
But the main issue was state or “drift management”, the desired state vs existing state was harder to ensure …
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UizTbB5XLFS-pY1A.png"><figcaption>Kubernetes | Controller Pattern</figcaption></figure>The Kubernetes controller pattern and its extensible design in the core of any modern application, Kubernetes attempts to abstract the application from the infrastructure nonetheless requires us to learn and adopt new skills, and for a professional to understand the patterns.
Skilling Up the Dev in DevOps? Considering we've been around the DevOps block for a while and are familiar with the requirements, it is now something we need to get our job done better, and with much less frustration …
<figure><img alt="" src="https://cdn-images-1.medium.com/max/256/0*hp9LLNZ_LDtsuwWX"><figcaption>Come skill-up with us <a href="https://www.experts.tikalk.com/">https://www.experts.tikalk.com/</a></figcaption></figure>Thanks for making it to the line ;).
HP
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=f601d95035f8" width="1" height="1" alt=""><hr>
<a href="https://medium.com/everything-full-stack/upskillin-the-dev-in-devops-f601d95035f8">👨🏼‍🔬 &gt; 🥷🏼 | UpSkillin’ the Dev in DevOps</a> was originally published in <a href="https://medium.com/everything-full-stack">Everything Full Stack</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

‍ &gt;  | UpSkillin’ the Dev in DevOps

<h3>There’s 🚫 place like 🏠 k3s — Meetup Notes</h3>
TLDR; You can’t avoid running your production’s “target os” locally. Learn to love it ;) &amp; Treat it like you would treat IaaS/SaaS/PaaS computing models.
With our everlasting “need for speed” 🚀, we've shifted from a monolith / central approach to a distributed one, which also meant moving our workload to the cloud …☁️
It’s not the scale we're dealing with -&gt; that’s for another day, it’s with scheduling, or to put it in more precise terms the scheduler and considering this pattern is what we're using in production, and Kubernetes flavored scheduler should do the job …
In this meetup with @yanir from up9, We started by looking at k3s and their advantages for developers, how the abstraction of kubernetes and the consolidation of k3s/d allows us to schedule isolated processes locally, followed by Yanir’s presentation of two Open Source projects <a href="https://wiremock.org/">WirwMock</a> &amp; <a href="https://github.com/up9inc/mizu">Mitzu</a> answering the challenges that might come with running on it ... in our DevEnv.
<h4>
Cloud-Native thinking (&amp; What does that mean for developers ?)</h4>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*FDlfTH1c7QqHnoA6OuSVmw.png"><figcaption>“Architecture styles”</figcaption></figure>For operational reasons we’ve classified our workloads into a bunch of architectures, all of these have been commoditized into a few which have become the common “set of architectures” in a distributed application.
<ul>
<li>What does that mean for devs? In general, it means the regular tools will not suffice ;)</li>
<li>How to think in a cloud-native way? We could continue “holding on to the IDE” (let’s not go there ATM) We should adopt new tools &amp; techniques (the workload has moved, we haven’t …)</li>
</ul>
In a but’s shell:
<blockquote>Treat your local machine as IaaS</blockquote>
And if I extend on that Use the Best Arch to get the job done.
This statement means that on a given day I might:
<ul>
<li>switch architectures</li>
<li>use a different / custom IDE</li>
<li>switch between environments to debug (“remote debugging”)</li>
</ul>
<blockquote>
Context switching and Engineering don’t work well together ;) … ֿֿ</blockquote>
One of the problems of choosing A cloud (over another) is that we also need to educate ourselves about how they call certain `things` e.g FaaS — Function As a Service, which has 5–6 different commercial names … not fun to learn each time.
One big advantage of the virtualization and cloud is how we consume the four primitives cpu, ram, network, and disk in a continuous and consistent (predictable way) like we shop for a T-shirt in an online store (we can always return it if it doesn't fit …) — our data won't move without the four of them.
<h4>Cloud-Native, Self-service — “Workload Scheduling”</h4>
If we fast forward, what VM’s, containers, and cloud abstractions gave us, is the ability to treat infrastructure (what we need for development), as self-service, and hopefully comprehensive enough for the end-user.
The catalyst for that was the Cloud and not long after Kubernetes, the best-known tech (ATM..) which abstracts the workload from the compute/cloud provider …
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2eKJg0NQrWWIHup__csOXQ.png"><figcaption>consistent workload scheduling</figcaption></figure>With this approach, being able to run Kubernetes on your laptop — isn’t that far-fetched … unless you have a lightweight scheduler …
<h4>Cloud-Native Tools | k3d+k3s</h4>
As with any distributed system, there are many components and the wiring of them all is something we might spend days getting to work. part of the cloud-native thinking is to declare/inherit from the platform/infra capabilities …
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LSn_utr51xTZLwuEo3Vf7Q.png"><figcaption>k3s + k3d — you can run it on 127.0.0.1</figcaption></figure>k3s is a lightweight all-in-one kubernetes binary you can read all about <a href="http://k3s.io/">k3s</a> and <a href="http://k3d/">k3d</a> I would like to emphasize how easy it is to:
<ul>
<li>Create a DevEnv that is similar to your Staging / Production env [ same scheduler ]</li>
<li>1 command on your laptop</li>
<li>Kubernetes nodes == Containers ♾</li>
<li>Run a multi-node cluster on your laptop</li>
</ul>
<pre># omit --servers/--agents to get a single node cluster k3d cluster create demo-cluster --servers 1 --agents 2 --port 443:80@loadbalancer --wait</pre>
<ul>
<li>Use the same API you use to debug staging/production — kubectl
</li>
<li>Work with registries/registry mirrors</li>
<li>k3d tools (import images during development)</li>
</ul>
+Many more cool features &amp; a special thanks to RancherLabs and CNCF for maintaining this project.
<blockquote>Side note:</blockquote>
<blockquote>During the meetup we had yesterday, one of <a href="https://medium.com/u/c8181e2b5473">Yoram Michaeli</a> asked me why not showing kubernetes through the IDE. My immediate answer was, you can avoid other tools such as kubectl, kubectx/ns, + some visualization tool such as lens / Octant, and others which seem to have claimed the term IDE in the kubernetes namespace … nonetheless we need more to get s**t done !
</blockquote>
<h4>HandsOn / Take away</h4>
Some examples to take away from that meetup, credit to <a href="https://medium.com/u/f4cb33e4a07f">Thorsten</a> on his cool scripted tutorial … which surely helped to boost the demo, my version is available on Gitlab
<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*uJFu9CpQfjkoCcTNiEx_bA.png"></figure><h4>To conclude:</h4>
If ~10 years ago, I heard something like “your computer is a cloud” … not sure what my reaction was back then … I've surely come to an agreement on that ;).
With my synthetic sugar,
<blockquote>What we used to call 127.0.0.1 has now become kubernetes.cluster.local — just another layer on top … a different, better work experience.
</blockquote>
Full slides from the meetup
<a href="https://medium.com/media/8e7bc191edfc208731a1620447670da1/href">https://medium.com/media/8e7bc191edfc208731a1620447670da1/href</a>Thanks for taking the time to reach this line ;)
HP
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=3bd7824faa5c" width="1" height="1" alt=""><hr>
<a href="https://medium.com/everything-full-stack/theres-place-like-k3s-meetup-notes-3bd7824faa5c">There’s 🚫 place like 🏠 k3s — Meetup Notes</a> was originally published in <a href="https://medium.com/everything-full-stack">Everything Full Stack</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

There’s  place like  k3s — Meetup Notes

<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4s5okl17Q-OhSKKU.jpg"><figcaption>Large Scale Systems</figcaption></figure>Today even small startups may have to work with terabytes of data or build services that support hundreds of thousands of events per minute (or even a second!). By “scale”, we usually refer to a large amount of requests/data/events that the system should handle in a short time period.
Trying to implement services that need to handle large scale in a naive way is doomed to failure at worst, or to be very costly in the best-case scenario.
This article will describe a few principles and design patterns that enable systems to handle large scale. When we discuss large-scale (and mostly also distributed) systems we usually judge how good or stable they are by looking at three attributes:
<ul>
<li>
Availability: a system should be available as much as possible. Uptime percentage is key for customer experience, not to mention that an application is not useful if no one can use it. Availability is measured with <a href="https://en.wikipedia.org/wiki/High_availability#%22Nines%22">“nines”</a>.</li>
<li>
Performance: a system should continue to function and perform its tasks even under heavy loads. Furthermore, speed is critical for customer experience: experiments show that it’s one of the most important factors in preventing churn!</li>
<li>
Reliability: a system should accurately process data and return a correct result. A reliable system does not fail silently or return incorrect results or create corrupted data. A reliable system is architected in a way that puts effort into avoiding failures, and when it’s not possible it will detect, report, and maybe even try to auto-remediate them.</li>
</ul>
We can scale a system in <a href="https://en.wikipedia.org/wiki/Scalability#Horizontal_(scale_out)_and_vertical_scaling_(scale_up)">two ways</a>:
<ul>
<li>
Scale vertically (scale-up): deploy the system on a stronger server, meaning, a machine that either has a stronger CPU, more RAM or both</li>
<li>
Scale horizontally (scale-out): deploy the system on more servers, meaning, spin up more instances or containers that enable the system to serve more traffic or process more data/events</li>
</ul>
Scaling-up is usually less preferable mainly because of two reasons:
<ul>
<li>it usually requires some downtime</li>
<li>there are limits ( we can’t scale up “forever”)</li>
</ul>
On the other hand, in order to be able to scale a system out, it has to have certain characters that allow this kind of scaling. For example, in order to be able to scale horizontally, a system has to be <a href="https://whatis.techtarget.com/definition/stateless">stateless</a> (e.g. most databases can’t be scaled out).
This article’s intention is to give you a taste of many different design patterns and principles that enable systems to scale out while maintaining reliability and resiliency. Due to this nature, I wasn’t able to go deep into every topic but rather just provide an overview. That said, in each subject I tried to add useful links to more thorough resources on that subject.
So let’s dig in!
<h3>Idempotence</h3>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/972/0*lnt3h-q3CXDwpxXx.jpg"></figure>This term is borrowed from Mathematics, where it’s defined as:
<pre>f(f(x)) = f(x)</pre>
This may look intimidating at first but the idea behind it is simple: no matter how many times we call the function fon x we’ll get the same result. This property provides great stability to a system because it allows us to simplify code and it also makes our operational life easier: an HTTP request that fails can be retried, and a process that crashed can be restarted without worrying about side effects.
Furthermore, a long-running job can be split into multiple parts each of which can be idempotent on its own, meaning, when the job crashes and gets restarted, all the parts that already got executed will get skipped (resumability).
<h3>
Embracing Asynchronicity</h3>
<figure><img alt="Async" src="https://cdn-images-1.medium.com/max/1024/0*Wsie1Ljk-F14CMc_.jpg"></figure>When we make a synchronous call, the execution path is blocked until a response is returned. This blocking has an overhead of resources, mainly the cost of memory and context-switch. We can’t always design our systems using async calls only, but when we can we make our system more efficient. One example that shows how async provides good efficiency/performance is Nodejs, which has a single-threaded event-loop, yet it is giving a fight to many other concurrent languages and frameworks.
<h3>Healthcheck</h3>
<figure><img alt="Healthcheck" src="https://cdn-images-1.medium.com/max/1024/0*ewdaJaIkky50wJQG"></figure>This pattern is specific to microservices: every service should implement a /health route which should return very quickly after running quick sanity of the system. Assuming that everything’s okay it should return HTTP code 200 and if the service is malfunctioning it should return 500 Error. Now, we know that some errors will not be caught by a healthcheck but assuming that a system under stress will function poorly and become latent, it will also be reflected by the healthcheck which will become more latent as well and this can also help us identify that there’s an issue and auto-generate an alert that an on-call can pick-up. We may also choose to temporarily remove the node out of the fleet (see Service Discovery below) until it becomes stable again.
<h3>Circuit Breaker</h3>
<figure><img alt="Circuit Breaker" src="https://cdn-images-1.medium.com/max/1024/0*B8NZSI8PTVop8LdR.jpg"></figure>Circuit breaker is a borrowed term from the electricity domain: when a circuit is closed the electricity is flowing and when a circuit is open the flow stops.
When a dependency is not reachable, all the requests to it will fail. According to the <a href="https://en.wikipedia.org/wiki/Fail-fast">Fail Fast principle</a>, we’ll prefer our system to fail fast when we try to make the call and not wait until there’s a timeout. This is a great use-case for Circuit breaker design pattern: by wrapping a call to a function with a circuit-breaker, the circuit-breaker will identify when calls to a specific destination (e.g. a specific IP) fail, and will start failing the calls without really making them, allowing the system to fail-fast.
The circuit-breaker will hold a state (open/close) and will refresh its state by retrying an actual call every interval of time.
An implementation of circuit-breaker was introduced and widely adopted in Netflix’s <a href="https://github.com/Netflix/Hystrix">Hystrix</a> library and today is common in <a href="https://github.com/resilience4j/resilience4j">other libraries</a> as well.
<h3>Kill Switch / Feature Flag</h3>
<figure><img alt="Feature Flag" src="https://cdn-images-1.medium.com/max/940/0*JjltwcjNyoe7tDiM.jpg"></figure>Another common practice today is to perform “silent deploy” to new features. It is achieved by gating the feature with a conditional if that checks if the feature flag is enabled (or alternatively, by checking that the relevant kill-switch flag is disabled). This practice doesn’t provide a 100% guarantee that our code is bug-free but it does have the effect of reducing the risk of deploying a new bug to production. Further, in case we enabled the feature flag and we see new errors in the system, it’s easy to disable the flag and “go back to normal” which is a huge win from an operational perspective.
<h3>Bulkhead</h3>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/475/0*mt6tyXNXStzzkwoq.jpg"></figure>Bulkhead is a dividing wall or barrier between compartments a the bottom of a ship. Its job is to isolate an area in case there’s a hole at the bottom — to prevent water from flooding all the ship (it will flood only the compartment in which the hole was created).
The same principle can be applied to software by building it with modularity and segregation in mind. One example can be <a href="https://stackoverflow.com/questions/30391809/what-is-bulkhead-pattern-used-by-hystrix">thread pools</a>: when we create different thread pools for different components to ensure that a bug that exhausts all the threads in one of them — will not affect the others.
Another good example is ensuring that different microservices will not share the same DB. We also avoid sharing configuration: different services should have their own configuration settings, even if it requires some kind of duplication, in order to avoid a scenario where a configuration bug in one service affects a different service.
<h3><a href="https://microservices.io/patterns/server-side-discovery.html">Service Discovery</a></h3>
<figure><img alt="Service Discovery" src="https://cdn-images-1.medium.com/max/1024/0*INq03CBGRffMYTAf.jpg"></figure>In the dynamic world of microservices, where instances/containers come and go, we need a way to know when a new node joins/leaves the fleet. Service discovery (also called <a href="https://github.com/Netflix/eureka">Service registry</a>) is a mechanism that solves this problem by allowing nodes to register in a central location, like yellow pages. This way when service B wants to call service A, it will first call service discovery to ask for a list of available nodes (IPs) which it’ll cache and use for a period of time.
<h3>Timeouts, Sleep &amp; Retries</h3>
<figure><img alt="Timeouts, Sleep &amp; Retries" src="https://cdn-images-1.medium.com/max/253/0*T-n3b3ySa0Ec4eOL.png"></figure>Any network may suffer from transient errors, delays, and congestion issues. When service A calls service B the request may fail, and if a retry is initiated the second request may pass successfully. That said, it’s important to not implement retries in a naive way (loop) without “baking” into it a mechanism of delay (also called “sleep”) between reties. The reason for that is that we should be conscious about the called service: there may be multiple other services that call service B simultaneously, and if all of them will just keep retrying, the result will be a “retry storm”: service B will get bombarded with requests which may overwhelm it and bring it down. In order to avoid a “retry storm”, it’s common practice to use an <a href="https://en.wikipedia.org/wiki/Exponential_backoff">exponential backoff</a> retry mechanism which introduces an exponentially growing delay between retries and an eventual “time-out” which will stop any additional retry.
<h3>Fallbacks</h3>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*xCgS6w_iLD2I4YQ6.jpg"></figure>Sometimes we just need a “plan B”. Say we’re using a recommendation service in order to get the best and most accurate recommendations for a customer. But what can we do when the service goes down or becomes unreachable for a few moments?
We can have another service as a fallback: that other service may keep a snapshot of recommendations of all our customers, refresh itself every week and when it’s called all it needs to do is return the relevant record for that specific customer. This kind of information is static and easy to cache &amp; serve. These fallback recommendations are a bit stale, true, but it’s much better to have recommendations that are not perfectly up-to-date than not having anything at all.
A good engineer considers such options when architecting their system!
Note that a circuit-breaker implementation may include an option to serve fallbacks!
<h3>Metrics, Monitoring &amp; Alarms</h3>
<figure><img alt="Metrics, Monitoring &amp; Alerts" src="https://cdn-images-1.medium.com/max/1024/0*_El6M7iL5OxneCJS.png"></figure>When operating a large-scale system, it’s not a question of if the system will fail but rather a question of when will it fail: due to the high-scale, even something so rare that can happen only once in a million events — will eventually happen.
Now that we understood and accepted errors as “part of life”, we have to figure out the best ways to deal with them.
In order to have a reliable system that is also available, we need to be able to detect (<a href="https://www.bmc.com/blogs/mttd-mean-time-to-detect/">MTTD</a>) and fix (<a href="https://en.wikipedia.org/wiki/Mean_time_to_repair">MTTR</a>) errors quickly and for that, we need to gain <a href="https://en.wikipedia.org/wiki/Observability">observability</a> into our system. This can be achieved by publishing metrics, monitoring these metrics, and raising an alarm whenever our monitoring system detects a metric that “goes off”.
Google defines <a href="https://sre.google/sre-book/monitoring-distributed-systems/#xref_monitoring_golden-signals">4 metrics as golden signals</a>, but that doesn’t mean that we shouldn’t have other metrics published as well. We can categorize metrics into 3 buckets:
<ul>
<li>Business metrics: a metric that is derived from a business context, for example, we may publish a metric every time an order is placed, approved, or canceled</li>
<li>Infrastructure metrics: a metric that measures size/usage of part of our infrastructure, for example, we can monitor the CPU usage, memory, and disk space used by our application</li>
<li>Feature metrics: a metric that publishes information about a specific feature in our system. An example can be a metric that is published on an A/B test we’re running to provide insight about users that are allocated to different cells of the experiment</li>
</ul>
Small anecdote: back in the days I worked for Netflix, one of the things my team and I did, was to develop <a href="https://netflixtechblog.com/introducing-winston-event-driven-diagnostic-and-remediation-platform-46ce39aa81cc">Watson</a> to enable teams to have their services auto-healed from known scenarios by creating a programmatic <a href="https://en.wikipedia.org/wiki/Runbook">runbook</a>!
<h3>Rate-limiting</h3>
<figure><img alt="Rate-limiting" src="https://cdn-images-1.medium.com/max/1024/0*lz_V1bY09P6cvUiI"></figure><a href="https://en.wikipedia.org/wiki/Rate_limiting">Rate-limiting</a> or throttling is another useful pattern that helps reduce stress from the system.
There are 3 types of throttling:
<ul>
<li>
<a href="https://cloud.google.com/architecture/rate-limiting-strategies-techniques#client-side_strategies">User rate-limiting</a> (client-side)</li>
<li>
<a href="https://cloud.google.com/architecture/rate-limiting-strategies-techniques#techniques-enforcing-rate-limits">Server rate-limiting</a> and</li>
<li><a href="https://docs.aws.amazon.com/waf/latest/developerguide/waf-rule-statement-type-geo-match.html">geographic rate-limiting</a></li>
</ul>
<h3>Backpressure</h3>
<figure><img alt="Backpressure" src="https://cdn-images-1.medium.com/max/500/0*t2x_5KiEvzCgQYCx.jpg"></figure><a href="https://www.reactivemanifesto.org/glossary#Back-Pressure">Backpressure</a> is a technique <a href="https://npm.io/package/hystrix-too-busy">used</a> to handle a situation where a load of requests is coming from an upstream service at a rate that is higher than can be handled. One way of dealing with backpressure is signaling the upstream service that it should rate-limit itself.
There’s a dedicated HTTP response-code 429 “too many requests” that’s designed to signal the client that the server is not ready to accept more requests at the current rate. Such a response is usually returned with a Retry-After <a href="https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html">header</a> to indicate how many seconds the client should wait before it retries.
Two other ways to handle backpressure are throttling (also known as “throwing requests on the floor”) and buffering.
Additional recommended reading about backpressure can be found <a href="https://www.infoq.com/news/2019/10/reactiveconf-2019-backpressure/">here</a>.
<h3>Canary release</h3>
<figure><img alt="Canary release" src="https://cdn-images-1.medium.com/max/430/0*auw4maCOEgqDajRR.jpg"></figure><a href="https://martinfowler.com/bliki/CanaryRelease.html">Canary testing</a> is a technique used to roll out changes gradually to production. When the monitoring system catches an issue — the canaries are rolled back automatically with minimal damage to production traffic.
Keep in mind that in order to enable canary release we need to be able to monitor the canary cluster separately from the “normal” nodes, then we can use the “regular” fleet of nodes as a baseline and compare it with the metrics we receive from the canaries. For example, we can compare the rate of 500 errors we receive in both and if the canaries produce a higher rate of errors we can roll it back.
There’s also a more conservative method of doing canaries using shadow traffic from production.
A previous colleague of mine Chris Sanden co-authored <a href="https://netflixtechblog.com/automated-canary-analysis-at-netflix-with-kayenta-3260bc7acc69">a good article</a> about <a href="https://github.com/spinnaker/kayenta">Kayenta</a>: a tool that was developed in Netflix for automated canary analysis.
That’s it for today folks, I hope you were able to learn something new!
If you think there’s an important pattern/principle I missed — please write a comment and I’ll add it.
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=f3c9adf89ad" width="1" height="1" alt=""><hr>
<a href="https://medium.com/everything-full-stack/design-patterns-and-principles-that-support-large-scale-systems-f3c9adf89ad">Design Patterns and Principles That Support Large Scale Systems</a> was originally published in <a href="https://medium.com/everything-full-stack">Everything Full Stack</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Design Patterns and Principles That Support Large Scale Systems

<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*W8jKDHtGmDW0NHwJqTJF5w.jpeg"></figure>Recently, I came across <a href="https://2ality.com/2022/01/pipe-operator.html#the-hack-pipe-operator">this article</a> about the ‘pipe operator for Javascript’ proposal from the <a href="https://tc39.es/proposal-pipeline-operator/#sec-ecmascript-language-expressions">TC39 group website</a>, and their <a href="https://github.com/tc39/proposal-pipeline-operator">GitHub</a>.
Although this article was well written and very detailed, I thought it was a little bit too long to read and requires a TL;DR hero to simplify it. So here comes your hero!
<h4>So what’s this proposal all about?</h4>
Actually, there were two proposals for the new pipe operator, both inspired by other programming languages. F# by Microsoft which was inspired by OCaml, and Hack by Facebook which was inspired by PHP. Both proposals do the same thing, but with different syntax.
Let’s start with a simple Javascript example:
<pre>const i = d(c(b(a)));</pre>
The a variable is applied to b, which applied to c, which applied to d, and all of this is assigned to i.
Now let’s use the new pipe operator:
<pre>|&gt;</pre>
This is how it will look like with Microsoft’s F#:
<pre>const i = a |&gt; b |&gt; c |&gt; d;</pre>
And with Facebook’s Hack:
<pre>const i = a |&gt; b(%) |&gt; c(%) |&gt; d(%)</pre>
You can see the main difference between the proposal syntaxes. While the F# is piping the variable to the next step, Hack needs you to use the % placeholder to define where the variable will be piped into.
So if we want to run a stringed number with parseInt, it’ll look like this:
<pre>'100' |&gt; parseInt // F# '100' |&gt; parseInt(%) // Hack</pre>
<h4>Looks the same, what’s the difference?</h4>
The Hack will be easier to use when we need to use methods, arrays, operators, objects or await:
<pre>value |&gt; $ =&gt; $.someMethod() // method call, value.someMethod() value |&gt; $ =&gt; $ + 1 // operator, value+ 1 value |&gt; $ =&gt; [$, 'b', 'c'] // Array literal, [value, 'b', 'c'] value |&gt; $ =&gt; {prop: $} // object literal, {prop: value} value |&gt; await // awaiting a Promise, await value</pre>
On the other side, F# will be easier with destructuring:
<pre>const str = obj |&gt; ({a, b}) =&gt; `${a} ${b}`;</pre>
<h4>What can I do with it?</h4>
Let’s see a very simple usage for the pipe operator:
<pre>// Hack const namesArray = arr |&gt; filter(%, obj =&gt; obj.enabled) |&gt; map(%, obj =&gt; objecy.name) |&gt; console.log(%) ; // F# const namesArray = arr |&gt; filter~(?, obj =&gt; obj.enabled) |&gt; map~(?, obj =&gt; obj.name) |&gt; console.log ;</pre>
in both examples, we can see how to pipe filter &amp; map and print the final result to console.log.
Of course, there are lots of <a href="https://github.com/tc39/proposal-pipeline-operator#real-world-examples">other use cases</a> the pipe operator can be very helpful, but I’ll leave you something to Google yourself :)
I hope this quick post was helpful for understanding the new pipe operator. Let me know what you think.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/150/1*2Ho-rgg1hbEzgsO5f1aruw.png"></figure><img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=664dcf1754ea" width="1" height="1" alt=""><hr>
<a href="https://medium.com/everything-full-stack/piping-your-js-664dcf1754ea">Piping your JS</a> was originally published in <a href="https://medium.com/everything-full-stack">Everything Full Stack</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Piping your JS

<figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*5V3zELy7wHj_lO3EBHrrkg.jpeg"></figure>So in my <a href="https://erezcarmel.medium.com/hooking-the-hooks-1724e85fa738">prior post</a>, I talked about how to create a hook of hooks.
This was a quick and helpful solution for what I needed. But it wasn’t the best solution, so I kept looking for an easier and more intuitive implementation for wrapping both 3rd party hooks packages with my project’s custom hooks.
The goal was to import hooks like this:
<pre>import { useOnline, useNetwork } from "./lib";</pre>
without using a hook to get the hooks altogether.
The solution I’ve found was to remove the useHooks custom hook I used before, and create a module that gathers all the hooks:
<pre>export * from "ahooks"; export * from "../hooks";</pre>
What you see here is pretty simple. I’ve exported all the ahooks package content, and all my custom hooks folder content, and served them as a new module called hooks .
The module is exporting both ahooks package and custom hooks folder and returns all the hooks together as a new module.
Now I can import all the hooks from one module, instead of importing different packages into my app, and I have one place to manage and control all my app’s hooks.
You can see the full working solution on my <a href="https://codesandbox.io/s/cranky-dijkstra-ozqcl">CodeSandBox</a>.
Let me know what you think.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/150/1*2Ho-rgg1hbEzgsO5f1aruw.png"></figure><img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=29feb889b69c" width="1" height="1" alt=""><hr>
<a href="https://medium.com/everything-full-stack/wrapping-your-hooks-29feb889b69c">Wrapping your hooks</a> was originally published in <a href="https://medium.com/everything-full-stack">Everything Full Stack</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Wrapping your hooks

<figure><img alt="" src="https://cdn-images-1.medium.com/max/297/1*R4X6BfnPTrko3Zy1FR3Itw.jpeg"></figure>Let’s say you have an external React hooks library you want to use, but also have your own custom hooks, how do you handle all your hooks altogether?
I come across this question during a React project I’m working on.
The development team I’m working with is willing to use the <a href="https://ahooks.js.org/">ahooks</a> library. A nice useful library that gives you tons of React hooks for your app development. Of course, they also develop their own custom hooks for their app.
The first question I asked them was “How do you plan to use hooks in your app?”
As I expected, they thought about importing the external library directly in their React components, and also importing their custom hooks at the same time.
Oh, no. Not on my shift.
When using external libraries in your app, keep in mind that you might replace it with a better, newer, fancy-schmancy library that will show up someday.
The right approach, in my opinion, is wrapping external libraries with internal util and exposing stable APIs for this util. That way, if you replace the external library, you don’t need to update 100’s of code implementation of this library across your app.
Back to our case, I decided to wrap all the hooks, both external and internal, in one internal hook:
<pre>import * as hooks from "ahooks"; import useOnline from "./use-online"; const useHooks = () =&gt; { return { ...hooks, useOnline }; }; export default useHooks;</pre>
I’ve created a useHooks hook and imported both ahooks and my own useOnline custom hook. The only thing my hook does is spreading ahook adding my custom hook, and return a new object with all the hooks altogether.
Pretty simple.
Now, anywhere in the React app, I don’t care which external hooks library we’ll use, the developers only need to use the useHook to get any custom hook:
<pre>const {useOnline, useToggle} = useHooks();</pre>
<pre>const [state, { toggle }] = useToggle(); const onlineStatus = useOnline();</pre>
In this example, the useToggle hook comes from the ahook library, but I also have my custom useOnline hook available.
That’s my 5 cents for today. Let me know what you think.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/150/1*2Ho-rgg1hbEzgsO5f1aruw.png"></figure><img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=1724e85fa738" width="1" height="1" alt=""><hr>
<a href="https://medium.com/everything-full-stack/hooking-the-hooks-1724e85fa738">Hooking the hooks</a> was originally published in <a href="https://medium.com/everything-full-stack">Everything Full Stack</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

About us

Careers

Radar

Home of
Tech Experts

We are a leading hands-on consultancy, scaling R&D teams with cutting-edge technologies

Why Our Customers Trust Us

Committed to make a Tech Impact

on your product

What We Do

Time to Scale?

Big Decisions Ahead?

Need More Skills?

We are Tikal - Full Stack and Beyond

Our Domains of Expertise

This is Our Story

On The Radar

What our experts have to say

Do you Waterproof your Software?

Terminal escape codes are awesome, here’s why

‍ UpSkillin’ the Dev in DevOps

Ready to Tech?

Company

Community

About us

Careers

Radar

Home ofTech Experts

We are a leading hands-on consultancy, scaling R&D teams with cutting-edge technologies

Why Our Customers Trust Us

Committed to make a Tech Impact

on your product

What We Do

Time to Scale?

Big Decisions Ahead?

Need More Skills?

We are Tikal - Full Stack and Beyond

Our Domains of Expertise

This is Our Story

On The Radar

What our experts have to say

Do you Waterproof your Software?

Terminal escape codes are awesome, here’s why

‍ UpSkillin’ the Dev in DevOps

Ready to Tech?

Company

Community

Home of
Tech Experts