# `MishkaCmsMigration.Source`
[🔗](https://github.com/mishka-group/mishka_cms_migration/blob/v0.0.1/lib/mishka_cms_migration/source.ex#L5)

A CMS a site is moved from: one module that is a `mishka_installer` plugin on the
`"migration:sources"` event and implements the callbacks below. WordPress is the first
(`MishkaCmsMigration.Sources.WordPress`); Joomla and others come the same way and reuse the
mapping, the dry run, the import job and the report.

    defmodule MyExtension.Sources.Ghost do
      use MishkaCmsMigration.Source, id: "ghost", label: "Ghost"
      use Gettext, backend: MishkaCmsCore.Gettext

      @impl true
      def kinds, do: ["user", "tag", "post"]

      @impl true
      def form_fields,
        do: [
          [
            name: :url,
            label: dgettext_noop("mishka_gervaz", "Address"),
            placeholder: "https://example.com",
            required: true
          ],
          [
            name: :secret,
            type: :password,
            label: dgettext_noop("mishka_gervaz", "Admin API key"),
            required: true
          ]
        ]

      @impl true
      def connect(%{"url" => url}, key), do: {:ok, Req.new(base_url: url, params: [key: key])}

      @impl true
      def list(req, kind, cursor) do
        ...
        {:ok, rows, next_cursor_or_nil}
      end

      @impl true
      def normalize(row, "post", _context),
        do: {:ok, %{source_id: row["id"], title: row["title"], content: [%{html: row["html"]}]}}
    end

`use` makes the module a plugin on `"migration:sources"` — registering is the declaration; the
event is never fired, and the plugin's `call/1` hands its state back — and adds `id/0`, `label/0`,
an `extra/1` that answers `%{}`, which a source overrides to keep facts about itself, a
`host/1` that reads the connection's `"url"`, a `form_fields/0` that asks for an address and a
secret, which a source overrides to ask for what it connects with, and a `file?/0` that answers
`false`, which a source that reads a file overrides. A source registers once it is supervised:
an extension that ships one lists the module among the children of its own application's
supervisor, as `MishkaCmsMigration.Application.plugins/0` lists this extension's. Which sources are installed is read from mishka_installer's own registry
(`installed/2`), so a source an operator stops in the installer is no longer offered and a run of
it is refused.

## A run

A run names its source by `id/0` (`MishkaCmsMigration.MigrationRun`'s `source`); a run of a source
that is not installed is refused when it is created. A job
(`MishkaCmsMigration.Workers.Helpers.Pipeline`) asks `connect/2` once, with the run's `connection`
map and its secret opened (`MishkaCmsMigration.MigrationRun.open_secret/1`), then walks `kinds/0`
in the order the source gives them, and pages through each with `list/3` until it answers no
cursor. The walk does not sort the kinds: a source lists them in the order they must be imported,
a kind before any kind that points at it, so authors come before the posts that name them. Each
listed row goes through `normalize/3`, which is handed the connection `connect/2` answered to read
more of the source when a row needs it (`t:context/0`), and then
`MishkaCmsMigration.Source.Item.build/1`. An error from `connect/2` or `list/3` fails the run; an
error from `normalize/3` fails only that item, and the report says why.

A job of picked items, Dry run this item or Import this item on the Items page, walks only the
kinds of those items and reads only their rows: `rows/3` with their source ids, where the source
has it, and otherwise every page of their kind from `list/3`, keeping the picked ones once they
are normalized. `rows/3` is optional; a source that reads a site over the network should have it,
so one item never costs a walk of its whole kind.

## Its connection form

The Migration page's New import opens one form for every source, and the form asks for what
`form_fields/0` names: the keys of the run's `connection`, as data, each the shape a Gervaz
`:key_map` key takes (`MishkaGervaz.Form.Types.Field.KeyMap`):

  * `:name` — the key the typed value is kept under in `connection`, as a string;
  * `:type` — `:text` (the default), `:password`, `:number`, `:select` or another type a
    `:key_map` key takes;
  * `:label` and `:placeholder` — msgids of the `mishka_gervaz` gettext domain, marked with
    `dgettext_noop("mishka_gervaz", "...")`, which the form translates as it draws them;
  * `:required` — `true` marks the key's label as a required field's label is marked.

The key named `:secret` is the secret the source signs in with. Give it `type: :password`: it is
drawn masked and never shown again, and creating the run takes it out of `connection` and keeps
it only sealed (`MishkaCmsMigration.MigrationRun.take_secret/2`), so `connect/2` gets it as its
`secret`. Every other key, a `:password` one too, is kept in `connection` as typed, so a source
has one secret, under `:secret`. A key left blank is not saved.

`use` asks for an Address (`:url`, required, `https://example.com` as its placeholder) and a
Password or key (`:secret`, a required `:password` key). `form_fields/1` reads the keys of the
source a form was opened for.

## A source that reads a file

A source that reads a file a site exported, rather than the site itself, answers `true` to
`file?/0` (`MishkaCmsMigration.Sources.WordPressFile` does) and names no keys in `form_fields/0`.
The New import form then asks for the file instead of a Connection, and the run keeps it in a
private folder until it ends (see "The source's file" in `MishkaCmsMigration.MigrationRun`):
`host/1` and `connect/2` get its path as the connection's `"file"`, and a refusal is shown on the
form's file. `file?/1` tells whether the source a form was opened for is one. `use` answers
`false`.

# `conn`

```elixir
@type conn() :: term()
```

What a source keeps between calls once connected: a client, a parsed file, a cursor.

# `context`

```elixir
@type context() :: %{
  site_id: String.t(),
  run_id: String.t(),
  source_host: String.t(),
  options: map(),
  conn: conn()
}
```

What `normalize/3` is told: the run's site, its source host and its options, and the connection
`connect/2` answered, for a row that names more of the source to read, such as the reusable
blocks and gallery images a WordPress post names.

# `cursor`

```elixir
@type cursor() :: term() | nil
```

Where `list/3` goes on from: `nil` is the first page, and answering `nil` ends a kind.

# `form_field`

```elixir
@type form_field() :: keyword()
```

One key of a source's connection form: `:name`, `:type`, `:label`, `:placeholder` and
`:required`, as a Gervaz `:key_map` key takes them (see "Its connection form").

# `connect`

```elixir
@callback connect(connection :: map(), secret :: String.t() | nil) ::
  {:ok, conn()} | {:error, term()}
```

Connects with a run's `connection` map and its opened secret (`nil` when it has none).

It is asked when a run is created, so a run is only kept for a source it can reach
(`MishkaCmsMigration.Resources.Changes.CheckConnection`), and again by each job. A refusal is
shown to the admin on the run's `connection`; these reasons are put in words, and any other is
shown as it is:

  * `:invalid_address` — the connection does not name an address the source can use;
  * `:unreachable` — nothing answered at the address;
  * `:no_site` — something answered, but not a site of this source;
  * `:unauthorized` — the site refused the user or the secret.

# `extra`

```elixir
@callback extra(conn()) :: map()
```

Facts about the source a run keeps in its `extra` (`MishkaCmsMigration.MigrationRun`), such as a
WordPress site's version and plugins. The job asks right after `connect/2`, before it walks a
kind, and the run keeps the answer when it finishes. `use` answers `%{}`.

# `file?`

```elixir
@callback file?() :: boolean()
```

Whether the source reads a file the admin uploads, handed to it as the connection's `"file"`,
rather than a site it connects to (see "A source that reads a file"). `use` answers `false`.

# `form_fields`

```elixir
@callback form_fields() :: [form_field()]
```

The keys the New import form asks for, in the order it draws them: what `connect/2` reads from a
run's `connection`, and its secret under `:secret` (see "Its connection form"). `use` asks for an
address and a secret.

# `host`

```elixir
@callback host(connection :: map()) :: String.t() | nil
```

The host a run's `connection` points at, kept as the run's `source_host` when the run does not
name one: with the source id and an item's id it is how a re-run finds what an earlier run
wrote. `use` answers the host of the connection's `"url"`, or `nil` without one.

# `id`

```elixir
@callback id() :: String.t()
```

The source's id, stored on a run.

# `kinds`

```elixir
@callback kinds() :: [String.t()]
```

The kinds of item this source lists, in the order they are imported: a kind before any kind that
points at it.

# `label`

```elixir
@callback label() :: String.t()
```

The source's name, as a `gettext_noop/1` msgid.

# `list`

```elixir
@callback list(conn(), kind :: String.t(), cursor()) ::
  {:ok, [term()], cursor()} | {:error, term()}
```

One page of `kind` from `cursor`, and the cursor of the next page (`nil` after the last).

# `normalize`

```elixir
@callback normalize(row :: term(), kind :: String.t(), context()) ::
  {:ok, map()} | {:error, term()}
```

A listed row as the map `MishkaCmsMigration.Source.Item.build/1` takes, without `:kind`, which
the walk adds.

# `rows`
*optional* 

```elixir
@callback rows(conn(), kind :: String.t(), source_ids :: [String.t()]) ::
  {:ok, [term()]} | {:error, term()}
```

The rows of `kind` whose source ids are `source_ids`, as `list/3` answers them, read without
listing the whole kind: what a job of picked items reads (see "A run"). A row it answers that was
not asked for is left out once it is normalized, so a kind the source reads in one go, such as
WordPress's menus, may answer it whole. Optional.

# `all`

```elixir
@spec all() :: [module()]
```

Every installed source (`installed/2`), by priority, the first of each id: what the Migration
page offers to migrate from, and what a developer reads to see which sources a node has.

# `connect`

```elixir
@spec connect(module(), map(), String.t() | nil, map()) ::
  {:ok, conn()} | {:error, term()}
```

Connects with `source` (`c:connect/2`) and answers what it answers: how the New import form's
check and each job connect. A refusal is also told on `[:mishka_cms_migration, :connect,
:refused]`, with `metadata`, the source's id and the reason in words (see "Telemetry" in
`MishkaCmsMigration.Workers`).

# `event`

```elixir
@spec event() :: String.t()
```

The event a source plugin registers on, the one `use MishkaCmsMigration.Source` names: what an
extension's own tests register a source for.

# `fetch`

```elixir
@spec fetch(String.t() | nil) :: {:ok, module()} | {:error, :not_installed}
```

The installed source whose id is `id`.

# `file?`

```elixir
@spec file?(map() | String.t() | nil) :: boolean()
```

Whether the installed source a form names (`defaults: %{source: id}`), or the source whose id is
given, reads a file (`c:file?/0`): what shows the New import form's file. `false` for a form
opened for no source and for a source that is not installed.

    field :file, :upload do
      visible fn state -> MishkaCmsMigration.Source.file?(state) end
    end

# `form_fields`

```elixir
@spec form_fields(map() | String.t() | nil) :: [form_field()]
```

The connection keys (`c:form_fields/0`) of the installed source a form names, or of the source
whose id is given: what the run's form draws under Connection. A form names its source in its
`defaults`, as the Migration page opens it (`defaults: %{source: id}`). Empty for a form opened
for no source and for a source that is not installed.

    field :connection, :key_map do
      options fn state -> MishkaCmsMigration.Source.form_fields(state) end
    end

# `installed`

```elixir
@spec installed(String.t(), module()) :: [module()]
```

The plugins of a migration event that are installed now, read from mishka_installer's own
registry (`MishkaInstaller.Event.Event.get/2`): those an operator has not stopped and that are
not held (`:started` or `:restarted`), by priority, that are loaded and implement every required
callback of `behaviour`, the first of each id. Nothing is fired and nothing is cached. Targets
are found the same way (`MishkaCmsMigration.Target.fetch/1`).

# `installed`

```elixir
@spec installed(String.t(), module(), String.t() | nil) ::
  {:ok, module()} | {:error, :not_installed}
```

The one plugin of `installed/2` whose `id/0` is `id`; `{:error, :not_installed}` for none.

# `label`

```elixir
@spec label(String.t()) :: String.t()
```

The name to show for the source whose id is `id`: its label in the reader's language, or the id
itself for a source that is no longer installed.

# `srcset`

```elixir
@spec srcset(String.t()) :: [{String.t(), String.t()}]
```

The candidates of an `srcset` attribute, each as `{url, descriptor}` (`{"hero-300.jpg", "300w"}`;
the descriptor is `""` for a candidate with none), in the order written. Candidates are told
apart by a comma followed by white space, or by one that follows a width or density descriptor.

# `srcset_value`

```elixir
@spec srcset_value([{String.t(), String.t()}]) :: String.t()
```

The `srcset` attribute `candidates` (`srcset/1`) written back as the attribute's value.

# `url_host`

```elixir
@spec url_host(map()) :: String.t() | nil
```

The host of a connection's `"url"`, lower-cased and with its port when it is not the scheme's
own (`localhost:8089`), or `nil` when it has no address with a host: what `use` answers as
`c:host/1`.

---

*Consult [api-reference.md](api-reference.md) for complete listing*