NHRC raises questions over Meta’s AI role in alleged CSAM promotion

New Delhi: The National Human Rights Commission (NHRC) has sought specific, point-wise Action Taken Reports (ATRs) from the Ministry of Electronics and Information Technology (MeitY), the Ministry of Information and Broadcasting (MIB), and Delhi Police over allegations that paid advertisements on Instagram promoted Child Sexual Abuse Material (CSAM) in India and redirected users to Telegram channels where such content was allegedly offered for sale.

The apex human rights body also sought an examination of whether Meta’s evolving role through AI and algorithmic systems could bring it within the regulatory purview applicable to publishers or online curated content under the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021.

The directions were issued by the NHRC after it considered the complaint and supplementary representation in the matter on Wednesday.

A Bench presided over by NHRC Member Priyank Kanoongo had earlier taken suo motu cognisance of media reports published by the BBC World Service alleging that paid advertisements on Instagram used expressions such as “rape video” and “child video” and redirected users to Telegram channels where such material was allegedly being sold.

The NHRC had earlier sought an ATR from the Delhi Police and, after the report was not submitted within the stipulated period, issued summons to the Delhi Police Commissioner, directing personal appearance before the apex human rights body on September 7.

The personal appearance was subsequently made subject to receipt of the requisite report by August 31.

In its latest directions, the NHRC said the allegations, if established, raise “grave concerns of online child sexual exploitation, circulation and monetisation of CSAM/CSEAM, possible organised criminal activity, and failures in intermediary due diligence, advertisement review, content moderation and child-safety mechanisms”.

The NHRC said the matter warrants examination under the Protection of Children from Sexual Offences (POCSO) Act, 2012, the Information Technology Act, 2000 and the applicable intermediary framework, including preservation and forensic scrutiny of electronic evidence, identification of advertisers and beneficiaries, tracing of financial trails and identification, rescue and rehabilitation of child victims.

It specifically referred to Section 19 of the POCSO Act, which mandates reporting of known or apprehended offences to the Special Juvenile Police Unit or local police. It observed that in cases involving suspected child sexual exploitation or circulation and commercial exploitation of such material, the statutory obligation “cannot be substituted by internal correspondence, grievance redressal or regulatory engagement”.

The NHRC said the Supreme Court in Just Rights for Children Alliance & Anr. vs. S. Harish & Ors judgment had clarified that Section 79 of the Information Technology Act does not, by itself, absolve intermediaries of their obligations under the POCSO Act and Rules.

The NHRC has now directed the MeitY Secretary to specifically state whether the Centre, upon acquiring knowledge of the allegations, reported or caused the matter to be reported to the Special Juvenile Police Unit or local police under Section 19 of the POCSO Act. If no such report was made, MeitY has been asked to identify the officer or authority responsible for ensuring mandatory reporting and indicate the action taken for any failure to report, including under Section 21 of the POCSO Act, as applicable.

The Centre has also been asked to examine whether Meta, in view of its evolving technological and functional role through systems capable of generating, editing, transforming, curating, recommending and disseminating content, falls within the regulatory purview of the MIB as an online content curator or publisher.

The NHRC has separately asked the MIB Secretary whether Meta, where its AI or algorithmic systems generate, modify, curate, recommend, publish or amplify content, falls within the regulatory purview of the Ministry as a publisher or publisher of online curated content under the IT Rules, 2021.

The directions came after a supplementary representation from the Network for Access to Justice and Multidisciplinary Outreach Foundation drew the Commission’s attention to Meta’s response to MeitY and raised questions concerning the changing role of digital platforms.

According to the representation, Meta’s AI-enabled tools were capable of suggesting content ideas, formats, slide sequencing, captions, calls-to-action, posting schedules, audience engagement strategies and monetisation methods.

The complainant organisation claimed that when the tool was tested in relation to a recent student-related issue, including a NEET-related matter, it suggested a structured content strategy for Instagram and Facebook, including a student-story post, a “Know Your Rights” carousel, a call-to-action post, posting schedules and monetisation options.

The NHRC said these assertions raise a “distinct regulatory question” as to whether a platform’s legal position requires separate examination where its own systems move beyond mere hosting or transmission and participate in the generation, modification, curation, recommendation, amplification or commercialisation of content.

It said such determination must depend upon “the actual function performed, degree of platform involvement/control and applicable law, rather than the mere availability of technological tools”.

The NHRC further observed that the issue assumes particular significance in cases involving children and CSAM/CSEAM, where AI-assisted creation, algorithmic recommendation, targeted amplification, advertising or platform-level curation could present risks distinct from passive transmission.

The Delhi Police Commissioner has been granted a further two weeks to furnish an updated ATR, specifically indicating the action taken in the matter and the information sought or received from Telegram, along with consequential action.

The NHRC has directed MeitY, MIB and Delhi Police to submit their respective specific, point-wise reports within two weeks.

The apex human rights body emphasised that the matter is “not merely about objectionable online content” but concerns the alleged commission of sexual offences against a child, their recording, circulation, promotion and monetisation through paid advertisements. It said that “behind such material is a child victim whose identification, rescue, protection and rehabilitation must remain paramount”.

IANS