For nearly every B2C company, revenue is linked to the customer lifecycle. Customer acquisition, engagement, and retention initiatives directly or indirectly impact revenue growth and sustainability by either reducing customer acquisition costs, increasing customer lifetime value (LTV), or ideally, both.

Effective marketing strategies revolve around the customer lifecycle; understanding key lifecycle stages, identifying events that are likely to trigger transitions between stages, and finding ways to optimize interactions across the lifecycle.

Naturally, the more you know about your customers and how to influence them to transition into a desired stage, the more efficiently you can grow and sustain revenue with the resources at your disposal.

Simply put, that's what customer lifecycle optimization is all about: leveraging rich customer data and predictive analytics techniques to generate insights and make predictions that measurably improve outcomes at each stage of the customer lifecycle.

## How customer lifecycle optimization works

Customer lifecycle optimization (CLO) is a practice. As with other practices, CLO involves a series of prescribed steps to be done effectively. While specific lifecycle stage names, data sources, and analytics techniques will vary depending on your industry, company, and objectives, the canonical CLO process comprises four steps: lifecycle mapping; data discovery; predictive groundwork; and implementation and action.

### Customer lifecycle mapping

The first step in any CLO initiative is always identifying and defining key lifecycle stages and transitions between stages. Mapping these stages and transitions to a uniform customer lifecycle is crucial to uncovering rich, predictive data.

The uniform B2C customer lifecycle stages are:

1. Lead generation
2. Lead conversion
3. Retention and expansion
4. Reactivation

While terminology will change from business to business, we found that this formulation is rich enough to capture important boundaries, yet simple enough to avoid stages with ambiguous transitions.

Think about your customer lifecycle. How are stages defined? Which attributes qualify individuals to be placed in those stages? What events trigger transitions between stages?

### Customer data discovery

At each stage, prospects, leads, and customers will complete certain events that will individually or collectively trigger a transition in or out of that stage. Individuals will also have different attributes, which help determine whether or not they belong in any given stage.

Go through each cell of your customer lifecycle map and think about which events or attributes in your data could be used to trigger a transition or pass the litmus test. Then, think about where you can find that data. Is it in your ESP, CRM, or a custom data warehouse?

### Customer prediction groundwork

Once you've formalized your lifecycle and its representation in data, you can start recognizing patterns and eventually predict outcomes. Imagine loading up your converted leads (customers in "retention and expansion") alongside your stale leads that never converted. What differences can you find?

This comparison analysis is especially effective when you've added depth and breadth to your existing data.

As you experiment with this type of pattern recognition, you'll quickly realize that it's the kind of thing that computers do very well. That's where machine learning comes in handy.

### Predictive engagement

Whether you want to increase conversion rates from lead generation campaigns or reduce churn from existing customers, properly targeted outreach is essential to engaging and motivating the right leads and customers to take a desired action.

Outbound communication is the strongest and most versatile intervention at an organization's disposal to compel progress and therefore expand revenue. This includes individually targeted digital advertising, a form of direct outreach.

Having identified a stage transition you'd like to motivate with an outreach intervention, the question becomes, "Who do I reach?" Regardless of the desired transition, the general technique is called audience expansion, also known as "lookalike" audiences.

To apply the audience expansion technique, we must always identify the audience and a set of candidates with similar characteristics and attributes.

With these groups defined, the next step is to apply your predictive groundwork. This could involve using patterns you identified in your data to look for similar opportunities among your candidates, or in more advanced cases, using artificial intelligence to build a predictive model trained to discriminate between likely and unlikely transitioners.

Finally, you will be left with a well-defined group of candidates likely to transition into a desired lifecycle stage when reached with relevant content.

## Putting it all together

To summarize the CLO practice, it's useful to recall the original motivation: leveraging rich data and statistical predictions to optimize revenue-building initiatives throughout the customer lifecycle. This means motivating transitions from one stage to the next.

Customer lifecycle optimization is the practice of leveraging data to maximize revenue-generating customer behaviors throughout the lifecycle. Learn how it works.

What is customer lifecycle optimization and how does it work?

**UPDATED FOR 2017** Now with easy subquery support and a more sensible argument order!

We adapted [this excellent piece by Dmitri Fontaine](http://tapoueh.org/blog/2014/02/21-PostgreSQL-histogram) and turned it into a function.

It can be invoked like this:

```
SELECT * FROM histogram($table_name_or_subquery, $column_name);
```

. . . to give sweet results like this, in a check of the distribution of 2016 political contributions in Vermont:

```
fec=# SELECT * FROM histogram('(SELECT * FROM small_donors_vt LIMIT 50000)', 'transaction_amt');

 bucket |   range   | freq |       bar
--------+-----------+------+-----------------
      1 | [0,9]     | 2744 | ******
      2 | [10,19]   | 5630 | *************
      3 | [20,29]   | 6383 | ***************
      4 | [30,39]   | 1290 | ***
      5 | [40,49]   |  369 | *
      6 | [50,59]   | 3541 | ********
      7 | [60,69]   |  174 |
      8 | [70,79]   |  313 | *
      9 | [80,89]   |  171 |
     10 | [90,99]   |   65 |
     11 | [100,109] | 2363 | ******
     12 | [110,119] |   51 |
     13 | [120,129] |  115 |
     14 | [130,139] |   32 |
     15 | [140,146] |   11 |
     16 | [150,159] |  187 |
     17 | [160,169] |   24 |
     18 | [170,177] |   33 |
     19 | [180,189] |   19 |
     20 | [191,199] |   24 |
     21 | [200,200] |  795 | **

```

Use it yourself by adding this to your postgres setup:

```
CREATE OR REPLACE FUNCTION histogram(table_name_or_subquery text, column_name text)
RETURNS TABLE(bucket int, "range" numrange, freq bigint, bar text)
AS $func$
BEGIN
RETURN QUERY EXECUTE format('
  WITH
  source AS (
    SELECT * FROM %s
  ),
  min_max AS (
    SELECT min(%s) AS min, max(%s) AS max FROM source
  ),
  histogram AS (
    SELECT
      width_bucket(%s, min_max.min, min_max.max, 20) AS bucket,
      numrange(min(%s)::numeric, max(%s)::numeric, ''[]'') AS "range",
      count(%s) AS freq
    FROM source, min_max
    WHERE %s IS NOT NULL
    GROUP BY bucket
    ORDER BY bucket
  )
  SELECT
    bucket,
    "range",
    freq::bigint,
    repeat(''*'', (freq::float / (max(freq) over() + 1) * 15)::int) AS bar
  FROM histogram',
  table_name_or_subquery,
  column_name,
  column_name,
  column_name,
  column_name,
  column_name,
  column_name,
  column_name
  );
END
$func$ LANGUAGE plpgsql;
```

Note:

1. You don't need to use a subquery - you can also just provide a table name as the first argument.
1. Subqueries are useful for sampling. For example, `(SELECT * FROM bigtable LIMIT 500 ORDER BY RANDOM()`. Don't forget parentheses!
1. You can call this as `SELECT histogram()` or `SELECT * FROM histogram()`. The latter form is much more legible!

Happy querying!

Learn how to create in-table histograms with PostgreSQL.

How to do histograms in PostgreSQL

Like many subscription services, meal delivery brand [Marley Spoon](https://marleyspoon.com/) faced the challenge of retaining customers. High churn rates made it difficult to cover acquisition costs and scale revenue, so they needed a way to identify risky customers. To improve their retention capabilities, Marley Spoon approached Faraday to assess the predictive accuracy between traditional segmentation and AI-driven churn modeling techniques.

## Traditional segmentation vs machine learning

With traditional methods, brands analyze trends among customer segments and take action to [prevent those with high churn rates](https://faraday.ai/predict/category/churn_scores) from unsubscribing. This is a relatively reactionary measure. In contrast, leveraging machine learning models to [score leads](https://faraday.ai/solutions/lead_management) and customers for their propensity to churn allows brands to segment customers based on churn risk and properly engage high-risk customers before they unsubscribe.

## Analysis and results

Because we’re most interested in the models’ ability to accurately predict churn, we evaluate performance based on how many churned customers were included in the top 10% and top 20% of predicted churners. Here are the results:

**Churn rate comparison:**

- 19% churn rate for likely churners identified with Faraday churn predictions
- 12% churn rate for likely churners identified through traditional segmentation

**Churn detection lift:**

- 70% lift over baseline with Faraday churn predictions
- 15% lift over baseline with traditional segmentation

To sum it up, Faraday's machine learning [predictions](https://faraday.ai/features/predictions) detected churn 5X better than the traditional technique.

Learn how the meal delivery subscription brand was able to identify likely churners 5X better with machine learning predictions.

How Marley Spoon predicted churn 5X better with machine learning

**Update** _We used to call this `jsonb_collect`, but then we realized it was very similar to `json_object_agg(name, value)`... except that there is no version of that function that just takes an expression. So, copying `jsonb_agg(expression)`, we give you..._

How do you aggregate (aka merge aka combine aka collect) JSONB fields in Postgres 9.5+? You define a new aggregate `jsonb_object_agg(expression)`:

```sql
CREATE AGGREGATE jsonb_object_agg(jsonb) (
  SFUNC = 'jsonb_concat',
  STYPE = jsonb,
  INITCOND = '{}'
);
```

Here's how you use it:

```console
# select * from greetings;
         data
-----------------------
 {"es":"Saludos"}
 {"en":"Hello"}
 {"ru":"Здравствуйте"}
(3 rows)

# select jsonb_object_agg(data) from greetings;
                        jsonb_object_agg
-------------------------------------------------------------
 { "es" : "Saludos", "en" : "Hello", "ru" : "Здравствуйте" }
(1 row)
```

If you're curious about the aggregate we just added, note that `jsonb_concat(jsonb, jsonb)` is the [function backing `||`](https://github.com/postgres/postgres/blob/REL9_5_STABLE/src/backend/utils/adt/jsonfuncs.c#L3349) that was [introduced in Postgres 9.5](https://www.postgresql.org/docs/9.5/static/release-9-5.html).

It's just that simple!

Learn how to aggregate (aka merge aka combine aka collect) JSONB fields in Postgres 9.5+.

How to aggregate JSONB in PostgreSQL 9.5+

At Faraday, we've long used [csvkit](https://csvkit.readthedocs.io/en/latest/) to understand, transform, and beat senseless our many streams of data. However, even this inimitable swiss army knife can be improved on - we've switched to [xsv](https://github.com/BurntSushi/xsv).

xsv is a fast CSV-parsing toolkit written in [Rust](https://www.rust-lang.org/en-US/) that mostly matches the functionality of csvkit (including the clutch ability to pipe between modules), with a few extras tacked on (like smart sampling). Did I mention it's fast? In a standup comparison, I ran the "stats" module of XSV against "csvstat" from csvkit, on a 30k-line, 400-column CSV file:

- Python-based csvkit chews through it in a respectable-and-now-expected 4m16s.

- **xsv takes 1.8 seconds.** I don't even have time for a sip of my coffee.

The difference between csvkit and xsv is partly defined by scale; both tools are plenty fast on smaller datasets. But once you get into 10MB-and-upward range, xsv's processing speed pulls away exponentially.

If you've been using csvkit forever (like me), or if you want to be able to transform and analyze CSVs without loading them into a DB, give xsv a shot:

### Install Rust

```bash
curl https://sh.rustup.rs -sSf | sh
```

. . . which also gives you the rust package manager `cargo`, which lets you:

### Install xsv

```bash
cargo install xsv
```

Then be sure your PATH is configured correctly:

```bash
export PATH=~/.cargo/bin:$PATH
```

. . . and try it out on a demo CSV with 10k rows, some messy strings, and multiple data types:

```bash
curl https://gist.githubusercontent.com/wboykinm/044e2af62fc0c7f77e17f6ccd55b8fb0/raw/fca391e6c03a06a7be770fefca6c47a9acdd2305/mock_data.csv \
| xsv stats \
| xsv table
```

_(`xsv table` formats the data so it's readable in the console):_

```bash
field           type     sum                 min                  max                  min_length  max_length  mean                stddev
id              Integer  5005000             1                    1000                 1           4           500.49999999999994  288.6749902572106
first_name      Unicode                      Aaron                Willie               3           11
last_name       Unicode                      Adams                Young                3           10
email           Unicode                      aadamsp5@senate.gov  wwrightd8@upenn.edu  12          34
gender          Unicode                      Female               Male                 4           6
ip_address      Unicode                      0.111.40.87          99.50.37.244         9           15
value           Unicode                      $1007.98             $999.37              0           8
company         Unicode                      Abata                Zoovu                0           13
lat             Float    243963.82509999987  -47.75034            69.70287             0           9           24.42080331331331   24.98767816017553
lon             Float    443214.19009999954  -179.12198           170.29993            0           10          44.36578479479489   71.16647723898215
messed_up_data  Unicode                      !@#$%^&*()           𠜎𠜱𠝹𠱓𠱸𠲖𠳏       0           393
version         Unicode                      0.1.1                9.99                 3           14
```

Happy parsing!

A data science hack to process CSVs faster using XSV.

How we made our CSV processing 142x faster

The Rust compiler just saved me from a nasty threading bug. I was working on an open source development tool for Docker apps with lots of microservices, and I decided to parallelize the routine that transformed `docker-compose.yml` files. This was mostly an excuse to check out the awesome [rayon][] library, but it turned into a great example of what real-world Rust development is like.

[rayon]: https://github.com/nikomatsakis/rayon

The original routine looked something like this:

```rust
/// Process our pods, flattening and transforming them using our
/// plugins, and output them to the specified directory.
fn output_helper(&self, op: Operation, export_dir: &Path) -> Result<()> {
    // Output each pod.
    for pod in &self.pods {
        // Don't export pods which aren't enabled.
        if !pod.enabled_in(&self.current_target) {
            continue;
        }

        // Figure out where to put our pod.
        // ...

        // Combine overrides, make it standalone, tweak as needed, and
        // output.
        let mut file = try!(pod.merged_file(&self.current_target));
        try!(file.make_standalone(&self.pods_dir()));
        let ctx = plugins::Context::new(self, pod);
        try!(self.plugins().transform(op, &ctx, &mut file));
        try!(file.write_to_path(out_path));
    }
    Ok(())
}
```

To convert this to a parallel loop, I started by changing:

```rust
for pod in &self.pods {
```

To:

```rust
self.pods.par_iter().map(|pod| -> Result<()> {
```

Here, `Result<()>` means "this closure might return an error, or it might return an empty tuple, basically `void`." (For more information on Rust error handling, check out [the Rust book][errors].) But it couldn't be that easy, could it?

[errors]: https://doc.rust-lang.org/book/ch09-00-error-handling.html

Nope. The Rust compiler showed me the following error message. It's long, but I'll translate:

```
error[E0277]: the trait bound `plugins::PluginTransform + 'static: std::marker::Sync` is not satisfied
   --> src/project.rs:397:30
    |
397 |         self.pods.par_iter().map(|pod| -> Result<()> {
    |                              ^^^ trait `plugins::PluginTransform + 'static: std::marker::Sync` not satisfied
    |
    = note: `plugins::PluginTransform + 'static` cannot be shared between threads safely
    = note: required because it appears within the type `Box<plugins::PluginTransform + 'static>`
    = note: required because of the requirements on the impl of `std::marker::Sync` for `std::ptr::Unique<Box<plugins::PluginTransform + 'static>>`
    = note: required because it appears within the type `alloc::raw_vec::RawVec<Box<plugins::PluginTransform + 'static>>`
    = note: required because it appears within the type `std::vec::Vec<Box<plugins::PluginTransform + 'static>>`
    = note: required because it appears within the type `plugins::Manager`
    = note: required because it appears within the type `std::option::Option<plugins::Manager>`
    = note: required because it appears within the type `project::Project`
    = note: required because it appears within the type `&project::Project`
    = note: required because it appears within the type `&&project::Project`
    = note: required because it appears within the type `[closure@src/project.rs:397:34: 424:10 op:&plugins::Operation, self:&&project::Project, export_dir:&&std::path::Path]`
```

The key bit to take away here is that "`Sync` is not satisfied" for the trait `plugins::PluginTransform`. In Rust, `Sync` is a special trait that tells the compiler that it's safe to share an object between two threads. That whole long list of "notes" afterwards looks intimidating, but it's really just telling me that my `PluginTransform` objects live inside my `plugins::Manager` type, which in turn lives inside my `Project`. The Rust compiler is verbose, but it tries to be helpful.

A bit of investigation reveals that `PluginTransform` inherits from the `Plugin` trait:

```rust
pub trait PluginTransform: Plugin {
```

So we can fix this problem by changing the declaration of `Plugin` from:

```rust
pub trait Plugin {
```

To:

```rust
pub trait Plugin: Sync {
```

This says, "All types implementing `Plugin` must also allow me to access them from multiple threads." With that fixed, I re-run `cargo test` and get a similar message:

```
error[E0277]: the trait bound `plugins::transform::vault::GenerateToken + 'static: std::marker::Sync` is not satisfied
   --> src/plugins/transform/vault.rs:230:6
    |
230 | impl plugins::Plugin for Plugin {
    |      ^^^^^^^^^^^^^^^ trait `plugins::transform::vault::GenerateToken + 'static: std::marker::Sync` not satisfied
    |
```

This time, I change:

```rust
trait GenerateToken: Debug {
```

To:

```rust
trait GenerateToken: Debug + Sync {
```

This says, "All types which implement `GenerateToken` must also implement `Debug` (so I can print them), as well as `Sync` (so I can share them). We're making progress!

Re-running `cargo test`, however, reveals the actual bug, and it would have been a nightmare to debug:

```
error[E0277]: the trait bound `std::rc::Rc<std::cell::RefCell<std::vec::Vec<(std::string::String, std::vec::Vec<std::string::String>, vault::client::VaultDuration)>>>: std::marker::Sync` is not satisfied
   --> src/plugins/transform/vault.rs:124:6
    |
124 | impl GenerateToken for MockVault {
    |      ^^^^^^^^^^^^^ trait `std::rc::Rc<std::cell::RefCell<std::vec::Vec<(std::string::String, std::vec::Vec<std::string::String>, vault::client::VaultDuration)>>>: std::marker::Sync` not satisfied
    |
```

Uh-oh. In our test harness, we have a type `MockVault`, which contains an `Rc<RefCell<_>>`:

```rust
type MockVaultCalls = Rc<RefCell<Vec<(String, Vec<String>, VaultDuration)>>>;

/// A fake interface to vault for testing purposes.
#[derive(Debug)]
#[cfg(test)]
struct MockVault {
    /// The tokens we were asked to generate.  We store these in a RefCell
    /// so that we can have "interior" mutability, because we don't want
    /// `generate_token` to be `&mut self` in the general case.
    calls: MockVaultCalls,
}

```

We use the `MockVault` to simulate a connection to Hashicorp's [Vault][], a secure central storage for passwords and other secrets, which issues time-limited credentials. And when we test our Vault code, we use the `calls` member here to record all the requests that we would have made to Vault.

[Vault]: https://www.vaultproject.io/

The type `Rc<RefCell<_>>` is a hack. The `GenerateToken` API assumes that our token-generator is a read-only object. This is good, because we want to access it from multiple threads! But in the test code, we need to create some "interior" mutable state. Basically, we ask Rust to replace compile-time borrow checks with run-time borrow checks. (For more details, see [the Rust book][mutability].) But `Rc<RefCell<_>>` is a lightweight mechanism designed for single-threaded code.

[mutability]: https://doc.rust-lang.org/beta/book/mutability.html

The fix is to replace `Rc<RefCell<_>>` with `Arc<RwLock<_>>`, which is fully thread-safe:

```rust
type MockVaultCalls = Arc<RwLock<Vec<(String, Vec<String>, VaultDuration)>>>;
```

Once we make this change, Rust reminds us to change all the code that accesses `calls`, too:

```
error: no method named `borrow_mut` found for type `std::sync::Arc<std::sync::RwLock<std::vec::Vec<(std::string::String, std::vec::Vec<std::string::String>, vault::client::VaultDuration)>>>` in the current scope
   --> src/plugins/transform/vault.rs:132:20
    |
132 |         self.calls.borrow_mut().push((display_name.to_owned(), policies, ttl));
    |                    ^^^^^^^^^^
    |
...
error: no method named `borrow` found for type `std::sync::Arc<std::sync::RwLock<std::vec::Vec<(std::string::String, std::vec::Vec<std::string::String>, vault::client::VaultDuration)>>>` in the current scope
   --> src/plugins/transform/vault.rs:368:23
    |
368 |     let calls = calls.borrow();
    |                       ^^^^^^
    |
```

Phew. If we hadn't caught that `Rc<RefCell<_>>`, our test suites might have randomly segfaulted a couple of times a week, and it would have taken us weeks to track it down. I don't even think our current test suites _could_ trigger this bug, but I bet a future version would have been able to, leaving a nasty surprise for us someday. The Rust compiler dug down through layers of data structures and found the one bit that wasn't thread safe.

## Wrapping it up

Rust also complains about that fact that we're calling `continue` from inside a closure:

```
error[E0267]: `continue` inside of a closure
   --> src/project.rs:402:17
    |
402 |                 continue;
    |                 ^^^^^^^^ cannot break inside of a closure
```

We can fix this by moving this code:

```rust
// Don't export pods which aren't enabled.
if !pod.enabled_in(&self.current_target) {
    continue;
}
```

...into a `filter` call on our parallel iterator:

```rust
self.pods.par_iter()
    // Don't export pods which aren't enabled.
    .filter(|pod| pod.enabled_in(&self.current_target))
    // Process each pod in parallel.
    .map(|pod| -> Result<()> {
```

The final problem is that we now have a parallel computation that returns a bunch of `Result<()>` objects, and we need to boil them down to a single `Result<()>` like we had before. If we have a bunch of error messages, I'm happy to just pick one. No need to overwhelm the user.

I spoke to Josh Stone, and he helped me come up with the following:

```rust
    .map(|pod| -> Result<()> {
        // ...
    }
    // If more than one parallel branch fails, just return one error.
    .reduce_with(|result1, result2| result1.and(result2))
    .unwrap_or(Ok(()))
```

This says, "Given any two results named `result1` and `result2`, take `result1` if it's an error, otherwise take `result2`." It's basically a short-circuit `&&` operator, but for `Result` values instead of booleans. I've filed an issue suggesting that `rayon` should provide those last two lines as a built-in function.

[issue113]: https://github.com/nikomatsakis/rayon/issues/113

And yes, that's a genuine parallel [map-reduce][] in Rust!

[map-reduce]: http://research.google.com/archive/mapreduce.html

So here's our final loop, using work-stealing parallelism:

```rust
fn output_helper(&self, op: Operation, export_dir: &Path) -> Result<()> {
    // Output each pod.  This isn't especially slow (except maybe the
    // Vault plugin), but parallelizing things is easy.
    self.pods.par_iter()

        // Don't export pods which aren't enabled.
        .filter(|pod| pod.enabled_in(&self.current_target))

        // Process each pod in parallel.
        .map(|pod| -> Result<()> {
            // Figure out where to put our pod.
            // ...

            // Combine overrides, make it standalone, tweak as needed, and
            // output.
            let mut file = try!(pod.merged_file(&self.current_target));
            try!(file.make_standalone(&self.pods_dir()));
            let ctx = plugins::Context::new(self, pod);
            try!(self.plugins().transform(op, &ctx, &mut file));
            try!(file.write_to_path(out_path));
            Ok(())
        })

        // If more than one parallel branch fails, just return one error.
        .reduce_with(|result1, result2| result1.and(result2))
        .unwrap_or(Ok(()))
}
```

This was pretty painless! I have a lot more things to parallelize, of course. But this is what day-to-day Rust development is like: I have to do a bit of extra work to satisfy the compiler (which mostly becomes a reflex). But in turn, the compiler ferrets out all kinds of subtle concurrency errors and generally watches my back. It's an interesting tradeoff, and overall I like it.

A detailed example of how to parallelize a loop with Rust and rayon.

Saved by the compiler: Parallelizing a loop with Rust and rayon

At Faraday, we use [redux-saga] in our client-side webapp because it offers a robust and testable set of tools for managing complex and asynchronous effects within Redux apps. I recently upgraded our version of redux-saga from an embarrassingly old version to the latest, in part to take advantage of a new (but largely undocumented) feature: context.

### The problem

I want to be able to write sagas without having to implicitly import any statically-declared singletons. The reason for that is simple: if I'm testing the saga that creates new users, I don't want to drag along the API layer or the router instance.

There are also a few modules in the codebase that _only_ work in a browser and will fail loudly when I try to run them in mocha. If one of these browser-only modules ends up in an import chain, suddenly my tests start failing for bad reasons. At worst, I have to rethink my testing strategy to include a mock browser environment. I'm lazy and hate testing, and I think mocks are a gross awful code smell that should be avoided at all costs.

### The solution

One of redux-saga's greatest features is that I can test a saga's behavior without actually executing that behavior, or even really knowing much about the details of how that behavior alters the world. It it should be possible to apply that to dependencies as well. And it is possible in redux-saga 0.15.0 and later using context.

Right now, there are very limited docs for context, but acts as shared value across all sagas that can be read with the `getContext` effect and written to by the `setContext` effect. It can also be set when the saga middleware is created by passing a context object as configuration.

### Example

Let's say we're writing an app that fetches game inventory data from the server using some kind of authentication. We don't want tokens and authentication to bleed into all our sagas, so we wrap it up in a nice singleton API service:

```js
class ApiService {
  getInventory = () => {
    return fetch("/api/inventory", {
      headers: {
        Authorization: `Bearer ${this.token}`,
      },
    }).then((res) => res.json());
  };
}

const api = new ApiService();
api.token = localStorage.token;

export default api;
```

Without context, our saga would probably statically import the API singleton:

```js
import { call } from "redux-saga/effects";
import api from "./api";

export function* fetchInventorySaga() {
  const inventory = yield call(api.getInventory);
  // Do something with the inventory data...
}
```

All is well until we try to test it in nodejs/mocha:

```js
import { fetchInventorySaga } from '../src/sagas/inventorySaga.js
// ReferenceError: localStorage is not defined
```

There is no localStorage in the nodejs global context. We can either pull in a testing harness to change how `import api from './api'` is resolved, attempt to run the tests in a browser, or roll our own late-binding mechanism so that you don't need to import API and can pass the API instance in at runtime.

We need to solve the same problem for `fetch`, because that's also absent in nodejs.

Or, we could use context. Our saga changes only a little bit:

```js
import { call, getContext } from "redux-saga/effects";
// No more API import:
// import api from './api'

export function* fetchInventorySaga() {
  // Get the api value out of the shared context:
  const api = yield getContext("api");

  const inventory = yield call(api.getInventory);
  // Do something with the inventory data...
}
```

We can now test the `getContext` effect just like we would any other redux-saga effect, and we can insert a mock value into `fetchInventorySaga` at test-time if we need to.

Setting up context in the main application is very straightforward. When you're creating your saga middleware:

```js
import createSagaMiddleware from "redux-saga";
import api from "./api";

const saga = createSagaMiddleware({
  context: {
    api, // our singleton API value
  },
});
```

Being able to late-bind singleton values like this has been enormously helpful writing robust tests in a complex codebase. I'll be steadily migrating the application code to use `getContext` more frequently, now that I have it as an option.

[redux-saga]: https://redux-saga.js.org
[a couple cryptic sentences]: https://github.com/redux-saga/redux-saga/releases/tag/v0.15.0

Redux-Saga offers a robust and testable set of tools for managing complex and asynchronous effects within Redux apps.

Using context in redux-saga

[Terraform's RDS support](https://www.terraform.io/docs/providers/aws/r/db_instance.html) makes it easy to create a database instance. Here's a cheatsheet:

```
resource "aws_db_instance" "mydb1" {
  allocated_storage        = 256 # gigabytes
  backup_retention_period  = 7   # in days
  db_subnet_group_name     = "${var.rds_public_subnet_group}"
  engine                   = "postgres"
  engine_version           = "9.5.4"
  identifier               = "mydb1"
  instance_class           = "db.r3.large"
  multi_az                 = false
  name                     = "mydb1"
  parameter_group_name     = "mydbparamgroup1" # if you have tuned it
  password                 = "${trimspace(file("${path.module}/secrets/mydb1-password.txt"))}"
  port                     = 5432
  publicly_accessible      = true
  storage_encrypted        = true # you should always do this
  storage_type             = "gp2"
  username                 = "mydb1"
  vpc_security_group_ids   = ["${aws_security_group.mydb1.id}"]
}
```

Here's the security group you need:

```
resource "aws_security_group" "mydb1" {
  name = "mydb1"

  description = "RDS postgres servers (terraform-managed)"
  vpc_id = "${var.rds_vpc_id}"

  # Only postgres in
  ingress {
    from_port = 5432
    to_port = 5432
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # Allow all outbound traffic.
  egress {
    from_port = 0
    to_port = 0
    protocol = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}
```

You can get these values from the EC2 console (don't forget them!):

```
variable "rds_vpc_id" {
  default = "vpc-XXXXXXXX"
  description = "Our default RDS virtual private cloud (rds_vpc)."
}

variable "rds_public_subnets" {
  default = "subnet-YYYYYYYY,subnet-YYYYYYYY,subnet-YYYYYYYY,subnet-YYYYYYYY"
  description = "The public subnets of our RDS VPC rds-vpc."
}

variable "rds_public_subnet_group" {
  default = "default-vpc-XXXXXXXX"
  description = "Apparently the group name, according to the RDS launch wizard."
}
```

Check out our Terraform RDS cheatsheet that makes it easy to create a database instance.

How to create an RDS instance with Terraform

(The more valuable and massive a data set is, the less likely it's in a format you can just parse. Has anybody else noticed that?)

Here's how to convert a fixed-width file to CSV with the [standard GNU unix tool `gawk`](https://www.gnu.org/software/gawk/):

### Theoretical (i.e., see "real life" below)

Thanks to [stackoverflow](http://stackoverflow.com/questions/28561786/convert-a-fixed-width-file-from-text-to-csv): (reproducing verbatim)

```
gawk '$1=$1' OFS=, FIELDWIDTHS='4 2 5 1 1' infile > outfile.csv
```

Where `FIELDWIDTHS` is a list of field widths and `OFS` is the output file separator.

### Real life

In real life, fixed width files contain commas and double quotes.

```
# put this in a file called fixed2csv.awk
{
  for (i=1;i<=NF;i++) {
    sub(/\s+$/,"",$i)
    sub("\"","\"\"",$i)
    printf "\"%s\"%s", $i, (i<NF?OFS:ORS)
  }
}
```

Then run it on your data:

```
gawk -f fixed2csv.awk OFS=, FIELDWIDTHS='4 2 5 1 1' infile > outfile.csv
```

Thanks to [Ed Morton on Stackoverflow](http://stackoverflow.com/questions/31947383/fixed-width-to-csv) for inspiration!

Learn how to convert a fixed-width file to CSV with the standard GNU unix tool gawk

How to convert a fixed-width file into CSV

Amazon's [explanation of deleting a versioned object](http://docs.aws.amazon.com/AmazonS3/latest/dev/RemDelMarker.html) and the [SDK documentation](https://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Object.html#delete-instance_method) do not give an example of permanently deleting a versioned object. Here's how to do it.

```ruby
require 'aws-sdk'

s3 = Aws::S3::Resource.new(
  region: 'us-east-1',
  access_key_id: ACCESS_KEY_ID,
  secret_access_key: SECRET_ACCESS_KEY
)
bucket = s3.bucket('my-versioned-bucket')

bucket.objects.each do |object_summary|
  o = bucket.object object_summary.key
  # this is the secret: specify the version while deleting
  o.delete version_id: o.version_id
end
```

If you don't specify the version, you get a delete marker, which you can proceed to delete infinite times and it will not go away :)

Learn how to permanently deleted version objects from AWS S3.

All posts