MishkaCmsMigration.Sources.WordPress (Mishka CMS Migration v0.0.1)

Copy Markdown View Source

The WordPress source ("wordpress"): a site read over its REST API, signed in to with a user name and an application password (WordPress 5.6+, Users ▸ Profile ▸ Application Passwords). WordPress offers application passwords over HTTPS only, or on a site whose WP_ENVIRONMENT_TYPE is local.

A run's connection names the site and the user; the application password is the run's secret:

%{"url" => "https://example.com", "user" => "admin"}

The New import form asks for them (form_fields/0): the Address ("url"), the User ("user") and the Application password (:secret, masked, which the run keeps only sealed), each required.

connect/2 first reads the REST index (/wp-json/), which tells it a WordPress answers at the address, then signs in (/wp-json/wp/v2/users/me?context=edit). It checks in that order, so the reason it gives is the first thing that is wrong. It refuses with one of the reasons MishkaCmsMigration.Source names:

  • :invalid_address — the address is not an http or https address with a host;
  • :unreachable — nothing answered at the address;
  • :no_site — something answered, but not a WordPress REST API: another kind of site, or a WordPress whose REST API a plugin turned off;
  • :unauthorized — no user or no password was given, or WordPress did not sign the user in. It answers a wrong password, an unknown user and a request whose Authorization header the host dropped the same way (rest_not_logged_in), so these cannot be told apart.

extra/1 keeps the site's name, its home address and timezone, and who signed in. When the REST index lists the REST namespace of a language plugin, Polylang's pll/v1 or WPML's wpml/v1, it keeps which one as "languages" ("polylang" or "wpml"): such a site names the language in its addresses, as /fa/about/, which the links an import writes do not follow (MishkaCmsMigration.Targets.Helpers.LanguageUrls).

What it reads

kinds/0 is ["user", "category", "tag", "media", "setting", "stylesheet", "page", "post", "comment", "post_redirect", "menu"]: authors, terms and files before the site's settings, its Elementor colours and fonts, and the pages and posts that point at them, the comments after the posts they are on, each post's old address after its post, and the menus after everything their links point at. "setting" is read from the REST index connect/2 already read, with no request of its own: the site's name, tagline (description), logo (site_logo) and icon (site_icon), each one a row when the site has it. "stylesheet" is the Elementor globals connect/2 read, with no request of its own: when the REST index lists Elementor's elementor/v1, connect/2 asks /elementor/v1/globals once, and a site that answers none has no row (MishkaCmsMigration.Sources.WordPress.ElementorKit). list/3 reads each other kind from its REST route (/wp/v2/users, /wp/v2/categories, /wp/v2/tags, /wp/v2/media, /wp/v2/pages, /wp/v2/posts, /wp/v2/comments) as the signed-in user, with context=edit, so WordPress answers the raw fields an editor sees, 100 rows a page; rows/3 asks the same route for the rows of picked items alone, by their ids. Pages and posts are asked for with status=any: published, scheduled, draft, pending and private, but not the trash. Comments are asked for with status=all and type=all: approved and held for moderation, of every type (WordPress answers only plain comments unless asked), but not spam and not the trash. "post_redirect" reads the posts again, only their id, title, status, link and modified_gmt (_fields). The cursor is the next page number while X-WP-TotalPages says there is one, and nil after the last. "menu" is one page: every menu (/wp/v2/menus, WordPress 5.9+) with its items (/wp/v2/menu-items) and every block menu (/wp/v2/navigation, asked for with status=any), each route read to its last page (menus/4). A list WordPress refuses fails the walk:

  • :unauthorized — WordPress answered 401 or 403: the user may not read this kind in the edit context, or the password was revoked since the run was made;
  • :unreachable — nothing answered;
  • :no_site — something answered that is not the REST API;
  • "WordPress answered <status>: <message>" — any other error WordPress put in words, such as a page past the last.

normalize/3 makes each row an item (MishkaCmsMigration.Source.Item). Its source_id is the WordPress id, and its parent the id of a page's parent page or a category's parent category (nil at the top, where WordPress says 0); keys are WordPress's own and are never atomized. WordPress keeps a slug written in another script percent-encoded (%d9%88%d8%b1… for وردپرس), so the item's slug is decoded; one that does not decode to text is kept as it is. WordPress saves a term's, a user's and a commenter's name as HTML, with &, < and > escaped (Tom &amp; Jerry), so each such name is read once as text (Tom & Jerry): the name of a category, a tag, a menu and a user, a user's first_name, last_name and nickname, and a comment's author_name, in the item's title and attrs alike.

  • page and post — title is title.raw, modified_at is modified_gmt as UTC, and content is content.raw converted (MishkaCmsMigration.Sources.WordPress.Content): its blocks, classic paragraphs and core shortcodes as HTML, and a piece for each block or shortcode left to migration:content:convert. The reusable blocks and gallery images it names are read from /wp/v2/blocks and /wp/v2/media with the connection the walk hands normalize/3 (its context's conn); without one, or when WordPress does not answer, they are pieces too. An entry laid out with Elementor is its layout instead (MishkaCmsMigration.Sources.WordPress.Elementor), and its links and files below are read from that content; the widgets in it the CMS has no equivalent for are named in its "unsupported_widgets", and those of them Elementor drew nothing for in its "no_fallback_widgets", which only such an entry has. attrs keeps slug, status, link, author, featured_media, date_gmt, template, excerpt (its raw text), and parent and menu_order for a page, categories, tags, sticky and format for a post. Its links are the links of its content to a post of the site, each href as written to the post's decoded slug. A link on the site's own host, or a path alone, is asked for: /?p=<id> by its id (include), any other by the last segment of its path (slug), from /wp/v2/posts in any status with only id, slug and link, and it is the post whose link has its path, or whose id it names. A link to a category, a file, an attachment page or a post the site no longer has is left out, and so is every link when the connection is not at hand. Its pages are the links of its content that are left, to a page of the site, each href as written to the page's "link" and "id" (%{"link" => "http://old.example/contact/", "id" => "29"}): /?page_id=<id> is asked for by its id and any other by the last segment of its path, the same way, from /wp/v2/pages, and it is the page whose link has its path, or whose id it names. Its media are the addresses of its converted content (src, href, a video's poster and every candidate of an srcset) that are a file of the site's media library, each as written to the id of the file it is: the original or one of the sizes WordPress made of it (hero-300x169.jpg is the file hero.jpg). The files are asked for from /wp/v2/media, with only id, source_url and media_details, by the ids of its wp-image-<id> images (include) and by slugs made of the file names, without a size (slug), and an address is a file's when its path is the path of the file or of one of its sizes. A file whose slug is not its name and that no wp-image-<id> names is left out, and so is every address when the connection is not at hand. Its seo is what Yoast SEO or Rank Math kept for it, each key only when it has one: "title", "description", "robots" (of noindex, nofollow, noarchive, nosnippet and noimageindex, joined by ", "), "image", the share image's address, with "image_id", the file of the media library it is, found with the content's files in the same request, and "canonical", only when it is on another host than the source's (every entry's own address is its canonical by default); and the social fields "og_title", "og_description", "twitter_title", "twitter_description", "twitter_image" (with "twitter_image_id", found like "image_id") and "twitter_card" (only summary, as the head draws summary_large_image by itself), each only when it says something else than the title or the description (or, for the Open Graph description, the entry's excerpt), which is what Yoast's head answers when nothing was written. Each is read from Rank Math's rank_math_* fields in meta, then Yoast's _yoast_wpseo_*, then the head Yoast answers (yoast_head_json), which is what the site served. An export carries every one of these fields; the REST API answers only a post's Yoast title and description among them, and none of Rank Math's, so the rest is read from Yoast's head: its title only when it is not the entry's own title, alone or followed by the site's name (Yoast's default, which the CMS's layout draws itself), and its robots only when they say noindex or nofollow. A variable in a title or description is the CMS's own where it has one (%%title%% and %title% as {{page.title}}, sitename as {{site.name}}, excerpt as {{page.description}}, sep as - and page as nothing), and a value with any other variable is left out. Each title and description is plain text, what the site showed: Yoast's head and a rendered excerpt hold HTML entities (Elementor&#8217;s), and so may a meta field, and each is read once (Elementor’s) before it is compared with the title, the description or the excerpt, so the CMS's head escapes it once.
  • media — title is title.raw and modified_at is modified_gmt as UTC; attrs keeps slug, source_url (where the file is), filename (the last segment of its path), mime_type, media_type, filesize (the file's bytes, media_details.filesize on a WordPress that answers none of its own), alt_text, caption and description (their raw text), post (the post it was uploaded to, 0 for none) and date_gmt. The files are listed oldest first (by id), so of two attachments that are the same file the earlier upload is the one the target meets first (MishkaCmsMigration.Targets.CmsMedia).
  • category and tag — title is name; attrs keeps slug, description, count, link, and parent for a category. A term has no modified_at.
  • user — title is name; attrs keeps username, email, slug, name, first_name, last_name, nickname, description, roles, url, link and avatar_urls. A user has no modified_at.
  • comment — title is the author's name, parent the id of the comment it replies to (nil at the top), and content the comment as its author typed it (content.raw); attrs keeps post (the id of the post or page it is on), post_type (that entry's type, "post" or "page", from the row's up link), author (the id of the WordPress user, 0 for a visitor), author_name, author_email, status ("approved" or "hold"; a spam or trashed comment, which a list never answers, reads as it is), type ("comment", "pingback" or "trackback") and date_gmt. A comment has no modified_at. Its address, website, user agent and the rest are kept in extra.
  • setting — one of the site's own: its source_id is "title", "tagline", "logo" or "icon", title WordPress's name for it ("Site Title", ...), and attrs's "value" the text of the name and the tagline, its HTML entities decoded (WordPress keeps & as &amp;), and the media id of the logo and the icon.
  • stylesheet — Elementor's global colours and fonts, one row: its source_id is "elementor", title "Elementor global colours and fonts", and attrs the stylesheet of theme tokens they become, its "name" (elementor-globals), "description" and "content" (MishkaCmsMigration.Sources.WordPress.ElementorKit.stylesheet/2).
  • post_redirect — the post's old address: its source_id is the post's id, title its title, and attrs's "from" the path of its link, such as /2025/12/hello/, for a published post with an address of its own; nil for any other, such as a draft, whose link is /?p=39, an address no redirect can answer.
  • menu — one menu: its source_id is the menu's id (navigation- and the post's id for a block menu, whose attrs say "block" and, when a classic menu has the same links, "same_as", that menu's name), title its name, and attrs keeps its slug, its theme locations, whether it is the site's main menu (menus/4), and its items, nested as WordPress nests them, each in menu order. An item is its label as text ("title"), the address WordPress gives it ("url"), what it points at as the kind and source id of an item of this source ("kind" and "id": a page, a post, a category or a tag, nil for any other), whether it opens in a new tab ("new_tab") and its own "items". A custom link of the main menu on the site's own host that names a post or a page, asked for as a post's links to them are, points at that post or page. An item WordPress counts as invalid, whose page or term is gone, is left out, and one whose parent is not in the menu sits at the top.

Each of these reads goes through the connection's read, a function of a REST route and its query answering the rows by id: connect/2 answers one that asks the site, and the export file source (MishkaCmsMigration.Sources.WordPressFile) one that answers from the file, which hands normalize/3 rows of the same shape, so both make the same items.

extra is every other field of the row: meta (Yoast's _yoast_wpseo_* fields and every other registered meta), guid, password, comment_status, class_list, a user's capabilities and registered_date, a term's taxonomy, a file's media_details (its sizes among them), and what a plugin adds. It leaves out what WordPress works out afresh for each answer rather than keeps: _links, the rendered halves of the fields above, and Yoast's yoast_head and yoast_head_json; and the local modified, which modified_at already says. It leaves out Elementor's own meta too (_elementor_*: the layout an entry's content is made of, its version, caches and CSS), which Elementor answers, empty, for every entry it could lay out. Yoast's head carries the modified time too, so keeping either would make an item that was only saved again look changed to MishkaCmsMigration.Source.Item.hash/1. A row without the fields its kind needs, such as one read with _fields, is refused as :unreadable.

Languages

A page, a post, a category and a tag read from a site that runs a language plugin carry the language they are in as the item's "language", and the other editions of them as its "translations", by language, each the source id of that edition, the item's own left out (%{"en" => "121"}), read from what the plugin adds to the row:

A code is lower-cased, with - for _. These fields are not kept in extra. Polylang's free plugin adds no field to the REST API, but it lists its languages, each with the address its content is served under, at its public /pll/v1/languages, read once when the run connects: a row with neither field is in the language whose address its link sits under, the longest one (/fa/ before /), such as fa for http://site/fa/about/, with no "translations", which only the site's export says. A row whose link sits under no language's address, and any row of a site without Polylang, names no language and has neither key. WPML's REST API answers its default language alone unless a request names another, so a site that runs WPML is read once for each language of its /wpml/v1/languages, with ?lang=<code>: its categories, tags, pages and posts. A site without WPML is read as before, with no extra request.

Files are written to the CMS's media library (MishkaCmsMigration.Targets.CmsMedia), the site's name and pictures to its settings (MishkaCmsMigration.Targets.CmsSetting), its Elementor colours and fonts to a stylesheet of its own (MishkaCmsMigration.Targets.CmsStylesheet), pages as the CMS's own pages (MishkaCmsMigration.Targets.CmsPage), the main menu as the links of the site layout's header (MishkaCmsMigration.Targets.CmsLayout), and users, categories, tags and posts into Mishka Blog where it runs: users as bloggers of the site's users, matched by email (MishkaCmsMigration.Targets.BlogBlogger), and MishkaCmsMigration.Targets.BlogCollection, MishkaCmsMigration.Targets.BlogTag, MishkaCmsMigration.Targets.BlogPost, their comments (MishkaCmsMigration.Targets.BlogComment), and each post's old address as a 301 to the post (MishkaCmsMigration.Targets.PostRedirect).

Requests go through Req. req_options in the application environment are merged into every request, which is how the suite replays what a real WordPress answered (test/fixtures/wordpress):

config :mishka_cms_migration, MishkaCmsMigration.Sources.WordPress,
  req_options: [plug: {Req.Test, MishkaCmsMigration.Sources.WordPress}]

Summary

Functions

Returns a specification to start this module under a supervisor.

Callback implementation for MishkaInstaller.Event.Hook.enable/0.

Callback implementation for MishkaInstaller.Event.Hook.get/0.

Optional. See MishkaInstaller.Event.Hook.health_check/0. Override to report health.

The rows list/3 answers for "menu", of the REST API's menus and menu-items (or rows of the same shape): each menu with its "items", the items whose menus is its id but those WordPress counts as invalid, and "main", whether it is the site's main menu: the one at the theme's primary location, else the first at any location, else the first made (the lowest id), which comes first, so an import writes it before any other menu. Then the block menus, the wp_navigation posts of navigation (MishkaCmsMigration.Sources.WordPress.Navigation), with "same_as", the name of the classic menu that has the same links, when one has.

Callback implementation for MishkaInstaller.Event.Hook.register/0.

Callback implementation for MishkaInstaller.Event.Hook.restart/0.

The rows of kind with the WordPress ids source_ids, what a job of picked items reads: asked for from the kind's REST route as list/3 asks for a page, by their ids (include), 100 ids a request. The settings, read from the REST index, and the menus, whose main one and block twins are told apart only among them all, are the picked rows of list/3's one page. WordPress refuses as list/3 says.

Callback implementation for MishkaInstaller.Event.Hook.start/0.

Callback implementation for MishkaInstaller.Event.Hook.stop/0.

Types

error_return()

@type error_return() ::
  {:error, [%{action: atom(), field: atom(), message: String.t()}]}

okey_return()

@type okey_return() :: {:ok, struct() | map() | module() | [any()]}

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

config()

@spec config() :: keyword()

config(key)

@spec config(atom()) :: any()

enable()

@spec enable() :: okey_return() | error_return()

Callback implementation for MishkaInstaller.Event.Hook.enable/0.

get()

@spec get() :: keyword()

Callback implementation for MishkaInstaller.Event.Hook.get/0.

health_check()

@spec health_check() :: :ok | {:degraded, term()} | {:error, term()}

Optional. See MishkaInstaller.Event.Hook.health_check/0. Override to report health.

on_dependency_error(error)

@spec on_dependency_error(term()) :: term()

Optional. See MishkaInstaller.Event.Hook.on_dependency_error/1. Override to react.

register()

@spec register() :: okey_return() | error_return()

Callback implementation for MishkaInstaller.Event.Hook.register/0.

restart()

@spec restart() :: okey_return() | error_return()

Callback implementation for MishkaInstaller.Event.Hook.restart/0.

rows(conn, kind, source_ids)

The rows of kind with the WordPress ids source_ids, what a job of picked items reads: asked for from the kind's REST route as list/3 asks for a page, by their ids (include), 100 ids a request. The settings, read from the REST index, and the menus, whose main one and block twins are told apart only among them all, are the picked rows of list/3's one page. WordPress refuses as list/3 says.

start()

@spec start() :: okey_return() | error_return()

Callback implementation for MishkaInstaller.Event.Hook.start/0.

start_link(args \\ [])

stop()

@spec stop() :: okey_return() | error_return()

Callback implementation for MishkaInstaller.Event.Hook.stop/0.

unregister()

@spec unregister() :: okey_return() | error_return()

Callback implementation for MishkaInstaller.Event.Hook.unregister/0.