用 GStreamer 使用 HLS 快速搭建直播系统

前提条件是要先安装 gstreamer， 我使用的是 macbook air, 具体的安装步骤不在这里赘述，
请参见官方文档 Installing GStreamer

快速开始

其实步骤很简单

1. 简单测试一下，确保你安装的 gstreamer 工作正常，并能从摄像头中读取视频

• 将摄像头视频显示出来

gst-launch-1.0 avfvideosrc device-index=1 ! \
video/x-raw,width=1920,height=1080,format=UYVY,framerate=30/1 ! autovideosink

注： 可以通过 gst-device-monitor-1.0 命令来察看你的摄像头的 device-index

2. 录制摄像头视频到 m3u8 和 ts 文件中

gst-launch-1.0 avfvideosrc device-index=1 ! x264enc ! h264parse ! hlssink2 max-files=10 location=./record_%05d.ts playlist-location=./playlist.m3u8

在 linux 系统中可使用

gst-launch-1.0 -v v4l2src device=/dev/video1 ! decodebin ! videoconvert ! omxh264enc ! h264parse ! hlssink2 max-files=10 location=./record_%05d.ts playlist-location=./playlist.m3u8 

3. 显示所录制的视频文件

gst-play-1.0 playlist.m3u8

• 远程直播可以通过 web server, 例如 nginx, apache 等
  这里使用 "python3 -m http.server" 启动一个测试的服务器
  在另外一台电脑上访问 http://ip:port/playlist.m3u8，就能看到直播的视频了

讲到这里就完了，就这么简单，如果你不想写程序，就不用往下看了
如果你有兴趣自己写程序来完成上述步骤，那我们可以继续讲讲相关的代码。

hlssink2 plugin

这里主要用到了 Gstreamer 的 hlssink2 插件， 其源代码参见 hlssink2 source code

hlssink2 与采用复用 MPEG-TS 流作为输入的旧 hlssink 不同，该元素采用基本音频和视频流作为输入并在内部处理复用。 这使得 hlssink2 能够就何时启动新片段做出更好的决策，并且还可以更好地处理输入流，而且如果其上游没有 encoder element, 还可以根据需要生成关键帧。

hlssink2 元素仅将 TS 片段文件 和 playlist 播放列表文件写入指定目录，它不包含实际的 HTTP 服务器来服务这些文件。 只需将外部网络服务器指向包含播放列表和片段文件的目录即可。

example

我们可以用 C++ 语言简单写一个例子，就三个文件

1. hls-exam.cpp // 测试 gstreamer hlssink2 的代码
2. pipeline_controller.h // 构建 gstreamer pipeline 的接口文件
3. pipeline_controller.cpp //构建 gstreamer pipeline 的实现文件

• 测试代码很简单 hls-exam.cpp

#include <chrono>
#include <thread>
#include "pipeline_controller.h"

int main(int argc, char *argv[]) {
    auto controller = std::make_unique<PipelineController>();
    controller->init(argc, argv);
    controller->start();
    std::this_thread::sleep_for(std::chrono::seconds(10));
    controller->pause();
    std::this_thread::sleep_for(std::chrono::seconds(10));
    controller->resume();
    controller->stop();
    controller->clean();
}

• 接口文件 pipeline_controller.h

#pragma once

#include <gst/gst.h>
#include <glib.h>
#include <string>
#include <map>

class PipelineController {
public:
    PipelineController();
    virtual ~PipelineController();
    int init(int argc, char *argv[]);
    int clean();

    int start();
    int stop();

    int pause();
    int resume();

private:
    bool create_elements();
    bool link_elements();
    void unlink_elements();

    GstElement* create_element(const std::string& factory, const std::string& name);
    int setup_elements();

    std::string m_video_source;
    std::string m_video_target;

    GstElement* m_source_element;
    GstElement* m_target_element;
    GstElement* m_tee_element;
    GstElement* m_enc_element;

    std::map<std::string, GstElement*> m_elements;
    
    GMainLoop* m_loop;
    GstElement* m_pipeline;
    GstBus* m_bus;
    gulong m_probe_id;
};

• 实现代码 pipeline_controller.cpp

#include <iostream>
#include <chrono>  // chrono::system_clock
#include <ctime>   // localtime
#include <sstream> // stringstream
#include <iomanip> // put_time
#include <string>  // string
#include <fmt/core.h>
#include <chrono>
#include <thread>
#include <gst/gst.h>
#include <glib.h>
#include "pipeline_controller.h"

#define PAD_NAME "video"
#define TIME_FMT "%Y%m%d%H%M%S"
#define DEBUG_TRACE(msg) std::cout << "[" \
    << time(NULL) <<","<< __FILE_NAME__ << "," << __LINE__ << "]\t"<< msg << std::endl


static const GstPadProbeType pad_probe_type = GST_PAD_PROBE_TYPE_BLOCK_DOWNSTREAM;

static uint32_t deleted_fragments = 0;

bool has_option(
    const std::vector<std::string_view>& args, 
    const std::string_view& option_name) {
    for (auto it = args.begin(), end = args.end(); it != end; ++it) {
        if (*it == option_name)
            return true;
    }
    
    return false;
}

std::string_view get_option(
    const std::vector<std::string_view>& args, 
    const std::string_view& option_name) {
    for (auto it = args.begin(), end = args.end(); it != end; ++it) {
        if (*it == option_name)
            if (it + 1 != end)
                return *(it + 1);
    }
    
    return "";
}

std::string get_time_str(
    const std::chrono::system_clock::time_point& timePoint, 
    const std::string& strPattern)
{
    auto in_time_t = std::chrono::system_clock::to_time_t(timePoint);

    std::stringstream ss;
    ss << std::put_time(std::localtime(&in_time_t), TIME_FMT);
    return fmt::format(fmt::runtime(strPattern), ss.str());
}

static void check_pads(GstElement *element) {
    GstIterator *iter = gst_element_iterate_pads(element);
    GValue *elem;
    
    while (gst_iterator_next(iter, elem) == GST_ITERATOR_OK) {
        gchar * strVal = g_strdup_value_contents (elem);
        DEBUG_TRACE("pad: " << strVal);
        free (strVal);
    }
    gst_iterator_free(iter);
}


static gboolean delete_fragment_callback(GstElement *element, const gchar *uri, gpointer user_data) {
    // Your custom logic for handling fragment deletion here.
    // In this example, we will simply print a message.
    DEBUG_TRACE(++deleted_fragments << ". Deleted fragment: " << uri);
    return TRUE;
}

static GstPadProbeReturn block_downstream_probe(GstPad *pad, GstPadProbeInfo *info, gpointer user_data) {
    // Block the downstream data flow by returning FALSE in the probe function.
    DEBUG_TRACE("blocking stream...");
    return GST_PAD_PROBE_OK;
}

PipelineController::PipelineController()
: m_loop(nullptr)
, m_pipeline(nullptr)
, m_bus(nullptr)
, m_probe_id(0) {
    DEBUG_TRACE("PipelineController construct");
}

PipelineController::~PipelineController()
{
    DEBUG_TRACE("PipelineController destruct");
}

int PipelineController::init(int argc, char *argv[]) {
    gst_init(&argc, &argv);
    DEBUG_TRACE("PipelineController init");
    const std::vector<std::string_view> args(argv, argv + argc);
    const std::string_view video_source_plugin = get_option(args, "-s");
    const std::string_view video_target_plugin = get_option(args, "-t");

    m_video_source = "videotestsrc";
    m_video_target = "hlssink2";

    if (!video_source_plugin.empty()) {
        m_video_source = video_source_plugin;
    }

    if (!video_target_plugin.empty()) {
        m_video_target = video_target_plugin;
    }
    create_elements();
    setup_elements();
    link_elements();
    return 0;
}
int PipelineController::clean() {
    DEBUG_TRACE("PipelineController clean");
   
    //gst_object_unref to free pipeline resources including all added GstElement objects
    gst_object_unref(m_pipeline);
    gst_object_unref(m_bus);

    return 0;
}

int PipelineController::start() {
    DEBUG_TRACE("PipelineController start");
    //check_pads(m_target_element);
    
    std::string dot_file = "video_pipeline";
    //set environment variable, such as export GST_DEBUG_DUMP_DOT_DIR=/tmp
    GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN_CAST(m_pipeline), GST_DEBUG_GRAPH_SHOW_VERBOSE, dot_file.c_str());

    DEBUG_TRACE("start playing...");
    gst_element_set_state(m_pipeline, GST_STATE_PLAYING);
    return 0;
}
int PipelineController::stop() {
    DEBUG_TRACE("stop playing...");
    gst_element_set_state(m_pipeline, GST_STATE_NULL);
    return 0;
}

int PipelineController::pause() {
    DEBUG_TRACE("pause playing...");
    m_probe_id = 0;
    // Get the source pad of hlssink
    GstPad *hlssink_pad = gst_element_get_static_pad(m_target_element, PAD_NAME);
    if(hlssink_pad) {
        DEBUG_TRACE("to block stream");
        m_probe_id = gst_pad_add_probe(hlssink_pad, GST_PAD_PROBE_TYPE_BLOCK_DOWNSTREAM, block_downstream_probe, NULL, NULL);
    }
    
    return m_probe_id;
}

int PipelineController::resume() {
    DEBUG_TRACE("resume playing...");
    if (!m_probe_id) {
        DEBUG_TRACE("have not paused");
        return -1;
    }
    GstPad *hlssink_pad = gst_element_get_static_pad(m_target_element, PAD_NAME);
    if(hlssink_pad) {
        DEBUG_TRACE("to unblock stream");
        gst_pad_remove_probe(hlssink_pad, m_probe_id);
        m_probe_id = 0;
    }
    
    return 0;
    
}


bool PipelineController::create_elements() {
    DEBUG_TRACE("PipelineController create_elements");
    m_pipeline = gst_pipeline_new("video-pipeline");
    m_bus = gst_element_get_bus(m_pipeline);
    m_source_element = create_element(m_video_source, "video-source");
    m_tee_element = create_element("tee", "video-tee");
    m_enc_element = create_element("x264enc", "video-encoder");
    m_target_element = create_element(m_video_target, "video-target");

    if(m_source_element && m_tee_element && m_enc_element && m_target_element) {
        return true;
    }
    return false;
}

int PipelineController::setup_elements() {
    DEBUG_TRACE("PipelineController setup_elements");
    if (m_video_source == "videotestsrc") {
        g_object_set(m_source_element, "pattern", 0, NULL); // Set the test pattern
    } else if(m_video_source == "avfvideosrc") {
        g_object_set(m_source_element, "device-index", 0, NULL); // Set the test pattern
        
        GstCaps* caps = gst_caps_new_simple("video/x-raw",       
            "width", G_TYPE_INT, 1920,                               
            "height", G_TYPE_INT, 1080,                              
            "framerate", GST_TYPE_FRACTION, 30, 1, NULL);
             
        g_object_set(G_OBJECT(m_source_element), "caps", caps, nullptr);
        gst_caps_unref(caps); 
        
    } else {
        DEBUG_TRACE("unknown source element");
    }

    auto now = std::chrono::system_clock::now();
    std::string playlist_filename = get_time_str(now, "/tmp/playlist_{}.m3u8");
    std::string record_filename = get_time_str(now, "/tmp/record_{}_%05d.ts");

    DEBUG_TRACE("playlist filename: " << playlist_filename 
        << ", record_filename=" << record_filename);

    g_object_set(m_target_element, "location", record_filename.c_str(), NULL);
    g_object_set(m_target_element, "playlist-location", playlist_filename.c_str(), NULL);
    //g_object_set(m_target_element, "playlist-root", "/tmp", NULL);
    g_object_set(m_target_element, "playlist-length", 20, NULL);
    g_object_set(m_target_element, "max-files", 20, NULL);
    g_object_set(m_target_element, "target-duration", 10, NULL);

    g_signal_connect(G_OBJECT(m_target_element), "delete-fragment", G_CALLBACK(delete_fragment_callback), NULL);
    return 0;
}

bool PipelineController::link_elements() {
    DEBUG_TRACE("add elements");
    gst_bin_add_many(GST_BIN(m_pipeline), m_source_element, m_tee_element, m_enc_element, m_target_element, NULL);
    DEBUG_TRACE("link_elements");
    gst_element_link_many(m_source_element, m_tee_element, m_enc_element, m_target_element, NULL);
    
    return true;
}
void PipelineController::unlink_elements() {
    DEBUG_TRACE("unlink_elements");
    gst_element_unlink_many(m_source_element, m_tee_element, m_enc_element, m_target_element, NULL);

    DEBUG_TRACE("remove elements");
    gst_bin_remove_many(GST_BIN(m_pipeline), m_source_element, m_tee_element, m_enc_element, m_target_element, NULL);
  
}

GstElement* PipelineController::create_element(
    const std::string& factory, 
    const std::string& name) {
    DEBUG_TRACE("create_element:" << factory << ", name=" << name);
    GstElement* e = gst_element_factory_make(factory.c_str(), name.c_str());
    m_elements.emplace(std::make_pair(name, e));
    return e;
}

代码放置于 https://github.com/walterfan/gstreamer-cookbook
测试代码会产生

• 一个 playlist_20231019233046.m3u8 文件
• 若干个 /tmp/record_20231019233046_xxx.ts 文件

playlist_20231019233046.m3u8 的内容如下

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-ALLOW-CACHE:NO
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:10

#EXTINF:10,
record_20231019233046_00000.ts
#EXTINF:10,
record_20231019233046_00001.ts
#EXTINF:10,
record_20231019233046_00002.ts
...
#EXT-X-ENDLIST

控制台输出如下

./example/hls-exam
[1697729446,pipeline_controller.cpp,91] PipelineController construct
[1697729446,pipeline_controller.cpp,101]    PipelineController init
[1697729446,pipeline_controller.cpp,181]    PipelineController create_elements
[1697729446,pipeline_controller.cpp,252]    create_element:videotestsrc, name=video-source
[1697729446,pipeline_controller.cpp,252]    create_element:tee, name=video-tee
[1697729446,pipeline_controller.cpp,252]    create_element:x264enc, name=video-encoder
[1697729446,pipeline_controller.cpp,252]    create_element:hlssink2, name=video-target
[1697729446,pipeline_controller.cpp,196]    PipelineController setup_elements
[1697729446,pipeline_controller.cpp,219]    playlist filename: /tmp/playlist_20231019233046.m3u8, record_filename=/tmp/record_20231019233046_%05d.ts
[1697729446,pipeline_controller.cpp,233]    add elements
[1697729446,pipeline_controller.cpp,235]    link_elements
[1697729446,pipeline_controller.cpp,132]    PipelineController start
[1697729446,pipeline_controller.cpp,139]    start playing...
[1697729452,pipeline_controller.cpp,76] 1. Deleted fragment: /tmp/record_20231019233046_00000.ts
[1697729452,pipeline_controller.cpp,76] 2. Deleted fragment: /tmp/record_20231019233046_00001.ts
[1697729453,pipeline_controller.cpp,76] 3. Deleted fragment: /tmp/record_20231019233046_00002.ts
[1697729453,pipeline_controller.cpp,76] 4. Deleted fragment: /tmp/record_20231019233046_00003.ts
[1697729453,pipeline_controller.cpp,76] 5. Deleted fragment: /tmp/record_20231019233046_00004.ts
[1697729453,pipeline_controller.cpp,76] 6. Deleted fragment: /tmp/record_20231019233046_00005.ts
[1697729454,pipeline_controller.cpp,76] 7. Deleted fragment: /tmp/record_20231019233046_00006.ts
[1697729454,pipeline_controller.cpp,76] 8. Deleted fragment: /tmp/record_20231019233046_00007.ts
[1697729454,pipeline_controller.cpp,76] 9. Deleted fragment: /tmp/record_20231019233046_00008.ts
[1697729455,pipeline_controller.cpp,76] 10. Deleted fragment: /tmp/record_20231019233046_00009.ts
[1697729455,pipeline_controller.cpp,76] 11. Deleted fragment: /tmp/record_20231019233046_00010.ts
[1697729455,pipeline_controller.cpp,76] 12. Deleted fragment: /tmp/record_20231019233046_00011.ts
[1697729455,pipeline_controller.cpp,76] 13. Deleted fragment: /tmp/record_20231019233046_00012.ts
[1697729456,pipeline_controller.cpp,150]    pause playing...
[1697729456,pipeline_controller.cpp,155]    to block stream
[1697729456,pipeline_controller.cpp,82] blocking stream...
[1697729466,pipeline_controller.cpp,163]    resume playing...
[1697729466,pipeline_controller.cpp,170]    to unblock stream
[1697729466,pipeline_controller.cpp,144]    stop playing...
[1697729466,pipeline_controller.cpp,122]    PipelineController clean
[1697729466,pipeline_controller.cpp,96] PipelineController destruct