Mirroring Facebook pages to Bolt CMS

March 2022

As part of doing pretty much everything IT-related for friends and family, I built a website for a local group using Bolt CMS. It lets me declare a number of different content types instead of railroading me toward a single article/blog post table, and it’s easy to store all that configuration in a git repository.

Long story short, the group wanted to be able to automatically mirror their Facebook posts onto the website. One option, and the one they wanted at first, was a Facebook iframe embed, but that comes with a number of GDPR problems (you need a consent popup, and users who don’t accept will have part of the site missing) and design concessions (the embed is not stylable). Clearly, we needed to get Facebook posts into Bolt’s database.

The pages are public, so scraping was an option, but I felt it would be too fragile and high-maintenance. Besides, Facebook’s got to have some kind of API for this usecase, right?

Facebook bureaucracy

Turns out they do! But getting access to this API takes a ridiculous amount of steps. As expected, you need a Facebook account and some additional identity verification. Finding the actual endpoint to use and figuring out which kind of authentication token to use to access it months from the initial login also took some time, but that’s to be expected. Facebook’s API is huge!

Actually creating and maintaining an “app” (i.e. developer token) to use those APIs is much more complicated: Generating a new token, selecting which endpoints I want to use, and happily coding would be too easy. First, I needed to figure out that I need to create a “Business” app to have access to Facebook pages. Then, after a lot of clicking, the dev console prompted me to “verify” my app. At this point, I thought all the work was for naught, adapting my PHP script for external testing would involve far too much effort! But, it turns out, “verification” is just Meta-speak for having the dev click a bunch of checkboxes amounting to pinky-swearing to only ever use the API for good and subversive marketing. Meh.

Some weeks after I set all this up, Facebook started emailing me about various problems with my app, like that I had no public privacy policy link. Finally, they said they’d put my app in “Developer mode” until I fixed those issues. Looking at the documentation, this mode still allows users listed as testers to use the API, which suits me just fine. Bizarrely enough, the documentation also says “Business apps” can’t be in Developer mode. The dashboard now proclaims “API access deactivated due to enforcement”, which is weird, because the mirroring works just fine, and so do logins of users. I don’t have any desire to prod Facebook’s weird API policies any longer, so I guess I’ll deal with it when it breaks.

Coding, finally

Compared to getting (and retaining) API access, programming the mirroring task has been surprisingly easy. First, I added a “Facebook post” content type to Bolt:

# This is config/bolt/contenttypes.yaml
fb_posts:
    slug: fb_posts
    singular_slug: fb_posts
    icon_many: fa:facebook
    listing_sort: datetime
    sort: datetime
    records_per_page: 50
    fields:
        fbpage:
            type: text
        banner_image:
            type: image
        content:
            type: redactor
        fbid:
            type: text
        fblink:
            type: text
        slug:
            type: slug
            uses: content

I could then get to coding the main part. Bolt is based on Symfony, so I used its controllers, authentication system, YAML parser and HTTP client. The code is hopefully straightforward.

To facilitate mirroring periodically via a standalone PHP script, I split the code into an injectable service, which handles the actual mirroring, and a controller, which provides a “user interface” of sorts. I introduced three endpoints: /fbmirror, /fbmirror/login and /fbmirror/mirror:

The first just returns the config, just to make sure something’s not terribly wrong. The second redirects to a Facebook login page, gets a short-lived access token, and creates long-term page access tokens for each page listed in the config.

The third endpoint lets users manually trigger the mirroring process. It looks at all recent posts on configured Facebook pages, checks if they have been mirrored yet, and copies over the ones that haven’t. This means that later edits to posts won’t be picked up, but it lets the users make changes to mirrored posts (maybe add emphasis or other formatting).

All in all, figuring out how to insert Bolt content directly via PHP probably took the longest. Bolt’s internal/plugin API isn’t at all bad, it just needs much more documentation and more examples. For now, ripgreping the source code worked for me.

# This is config/services.yaml
# The service has to be explicitly declared as public, otherwise it can't
# be retrieved by a simple $container->get()
services:
    App\Service\FbMirrorService:
        public: true

<?php
// This is src/Controller/FbMirrorSettingsController.php
namespace App\Controller;

use App\Service\FbMirrorService;
use ApiPlatform\Core\Api\UrlGeneratorInterface;
use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\RedirectResponse;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Routing\Annotation\Route;

class FbMirrorSettingsController extends AbstractController {
    public function __construct(private FbMirrorService $service) { }

    private function checkOrDenyAccess() {
        $this->denyAccessUnlessGranted("ROLE_ADMIN");
    }

    /**
     * @Route("/fbmirror/login", name="fbmirror_login")
     */
    public function fbMirrorLogin(Request $request) {
        $this->checkOrDenyAccess();
        $config = $this->service->readConfig();
        $redirect_url = $this->generateUrl(
            'fbmirror_login', [], UrlGeneratorInterface::ABS_URL);

        if($request->query->has("error")) {
            return new Response("Error logging in! "
                . $request->query->get("error") . " "
                . $request->query->get("error_reason") . " "
                . $request->query->get("error_description"));
        }

        // TODO: Actually use the state param
        if($request->query->has("code")) {
            $log = "";

            $ttResp = $this->httpClient->request(
                "GET", "https://graph.facebook.com/v12.0/oauth/access_token", [
                    "query" => [
                        "client_id" => $config["client_id"],
                        "client_secret" => $config["client_secret"],
                        "redirect_uri" => $redirect_url,
                        "code" => $request->query->get("code"),
                    ],
                ]);
            if($ttResp->getStatusCode() === 200) {
                $temp_token = $ttResp->toArray()["access_token"];
                $log .= "Temp token: " . $temp_token . "\n";

                $ltResp = $this->httpClient->request(
                    "GET", "https://graph.facebook.com/v12.0/oauth/access_token", [
                        "query" => [
                            "client_id" => $config["client_id"],
                            "client_secret" => $config["client_secret"],
                            "fb_exchange_token" => $temp_token,
                            "grant_type" => "fb_exchange_token",
                        ],
                    ]);
                $long_token = $ltResp->toArray()["access_token"];
                $log .= "Long-lived token: " . $long_token . "\n";
                $log .= "\n";

                foreach($config["pages"] as &$page) {
                    $pageResp = $this->httpClient->request(
                        "GET", "https://graph.facebook.com/" . $page["id"], [
                            "query" => [
                                "fields" => "access_token",
                                "access_token" => $long_token,
                            ],
                        ]);
                    if($pageResp->getStatusCode() !== 200) {
                        $log .= "Getting token for page {$page["name"]} failed: "
                            . $pageResp->getStatusCode() . " "
                            . $pageResp->getContent(false) . "\n";
                        continue;
                    }
                    $page["access_token"] = $pageResp->toArray()["access_token"];
                    $log .= "Successfully got token for page {$page["name"]}\n";
                }

                $this->service->writeConfig($config);
                $log .= "\nFinished, config updated\n";

                return new Response($log, 200, [
                    "Content-Type" => "text/plain;charset=utf8"
                ]);
            }
        }

        return $this->redirect(
            "https://www.facebook.com/v12.0/dialog/oauth"
            . "?client_id=" . $config["client_id"]
            . "&redirect_uri=" . $redirect_url
            . "&scope=pages_read_engagement"
            . "&auth_type=rerequest"
            . "&state=TODO");
    }

    /**
     * @Route("/fbmirror", name="fbmirror_status")
     */
    public function fbMirrorStatus() {
        $this->checkOrDenyAccess();

        return new Response(
            "Config: " . print_r($this->service->readConfig(), true),
            200, ["Content-Type" => "text/plain;charset=utf8"]);
    }

    /**
     * @Route("/fbmirror/mirror", name="fbmirror_mirror")
     */
    public function fbMirrorMirror() {
        $this->checkOrDenyAccess();
        $log = $this->service->mirror();
        return new Response($log, 200, [
            "Content-Type" => "text/plain;charset=utf8"
        ]);
    }
}

<?php
// This is src/Service/FbMirrorService.php
namespace App\Service;

use App\Caching;
use Bolt\Entity\Content;
use Bolt\Configuration\Config;
use Doctrine\ORM\EntityManagerInterface;
use Symfony\Component\HttpKernel\KernelInterface;
use Symfony\Component\Yaml\Yaml;
use Symfony\Contracts\HttpClient\HttpClientInterface;

class FbMirrorService {
    public function __construct(
            private KernelInterface $kernel,
            private HttpClientInterface $httpClient,
            private EntityManagerInterface $entityManager,
            private Config $boltConfig) { }

    private function getConfigLocation() {
        return $this->kernel->getProjectDir() . "/var/fbmirror.yaml";
    }

    public function readConfig() {
        return Yaml::parseFile($this->getConfigLocation());
    }

    public function writeConfig(array $config) {
        file_put_contents($this->getConfigLocation(), Yaml::dump($config));
    }


    public function mirror() {
        $config = $this->readConfig();

        $log = "Mirroring posts from Facebook...\n";

        foreach($config["pages"] as $page) {
            if(!array_key_exists("access_token", $page)) {
                $log .= "Skipping {$page['name']}, no access token found...";
                continue;
            }

            $log .= "\nMirroring page {$page["name"]}...\n";
            $feedResp = $this->httpClient->request(
                "GET", "https://graph.facebook.com/{$page["id"]}/feed", [
                    "query" => [
                        "access_token" => $page["access_token"],
                        "fields" => "id,attachments,created_time,"
                                  . "full_picture,message,permalink_url",
                    ],
                ]);
            if($feedResp->getStatusCode() !== 200) {
                $log .= "Getting feed failed: "
                    . $feedResp->getStatusCode() . " "
                    . $feedResp->getContent(false) . "\n";
                continue;
            }

            $log .= "Successfully retrieved feed!\n";
            $feed = $feedResp->toArray();

            $existingBoltPosts =
                $this->entityManager
                     ->getRepository(Content::class)
                     ->findBy(["contentType" => "fb_posts"]);
            foreach($feed["data"] as $post) {
                $exists = false;
                foreach($existingBoltPosts as $existingBoltPost) {
                    if($existingBoltPost->getFieldValue("fbid") == $post["id"]) {
                        $exists = true;
                        break;
                    }
                }
                if($exists) {
                    $log .= "Post {$post['id']} already exists, skipping...\n";
                    continue;
                }

                if(!array_key_exists("message", $post)) {
                    $log .= "Skipping post {$post['id']}, "
                          . "because it is missing a message\n";
                    continue;
                }

                $boltPost = new Content();
                $boltPost->setContentType("fb_posts");
                $boltPost->setDefinitionFromContentTypesConfig(
                    $this->boltConfig->get('contenttypes'));
                $datetime = \DateTime::createFromFormat(
                    \DateTimeInterface::ISO8601, $post["created_time"]);
                $boltPost->setCreatedAt($datetime);
                $boltPost->setModifiedAt($datetime);
                $boltPost->setPublishedAt($datetime);
                $boltPost->setFieldValue("fbpage", $page["name"]);
                $boltPost->setFieldValue("fbid", $post["id"]);
                $boltPost->setFieldValue("fblink", $post["permalink_url"]);
                $boltPost->setFieldValue("content", $post["message"]);

                if(array_key_exists("full_picture", $post)) {
                    $image_path = "fb_banner_images/{$post['id']}.jpg";
                    file_put_contents(
                        $this->kernel->getProjectDir()
                        . "/public/files/"
                        . $image_path,
                        fopen($post["full_picture"], "r"));
                    $boltPost->setFieldValue("banner_image", [
                        "media" => "",
                        "alt" => "",
                        "filename" => $image_path
                    ]);
                }

                $this->entityManager->persist($boltPost);

                $log .= "Saved post {$post['id']}\n";
            }
        }
        $this->entityManager->flush();

        $log .= "Finished mirroring!\n";

        Caching::clearCache();
        $log .= "Flushed cache!\n";
        return $log;
    }
}

#!/usr/bin/env php
<?php
// This is cron/fbmirror.php
// I basically copied public/index.php, but changed it to run the mirroring
// process instead of processin the request..

use App\Service\FbMirrorService;
use App\Kernel;
use Bolt\Configuration\Config;
use Symfony\Component\Dotenv\Dotenv;

require dirname(__DIR__).'/vendor/autoload.php';

// Set the `time_limit` and `memory_limit`, if we're allowed to
set_time_limit(0);
if (Config::convertPHPSizeToBytes(ini_get('memory_limit')) < 1073741824) {
    @ini_set('memory_limit', '1024M');
}

(new Dotenv())->bootEnv(dirname(__DIR__).'/.env');

$kernel = new Kernel($_SERVER['APP_ENV'], (bool) $_SERVER['APP_DEBUG']);
$kernel->boot();
$container = $kernel->getContainer();
$service = $container->get(FbMirrorService::class);
$log = $service->mirror();
echo $log;