7hv %SSKJr SSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK r SSK r SSK r SSK r SSK r SSKJrJr SSKJrJr SSKJr SSKJrJrJrJrJrJrJrJrJr SSKJrJ r SSK!r!SSK"r"SSK#r"SSK$J%r% SS K&J'r( SS K)J*r* SS K+J,r, SS K-J.r/ SS K0J1r1J2r2J3r3 SSK4J5r5J6r6 SSK7J8r8J9r9 SSK:J;r; SSKr> SSK?J@r@JArAJBrBJCrCJDrDJErEJFrFJGrGJHrHJIrIJJrJJKrK SSKLJMrMJNrNJOrOJPrPJQrQJRrR \(acSSKSJTrTJUrUJVrV SSK#JWrW SSKXJYrY SSKZJ[r[J\r\J]r]J^r^ SSK_J`r` SSKaJbrbJcrcJdrd SSKeJfrf \ "S5rg\\\c/\b4rh\i\frj\\k\!R4rm\krn\"RR\qS5rr\R"\q5rtGSSjru\R"S S!55rw"S"S#\R$5rx"S$S%\5ry\E"S&S'9"S(S)\y55rz"S*S+5r{\R"S,S-55r|\R"S.S/55r}\R"S0S155r~\R"S2S355r\R"S4S555r\\z\|\}\\~4r0rS6\S7'"S8S95r0rS:\S;'0rS<\S='GSGSS>jjr"S?S@\5rGSSAjrGSSBjrGSSCjrGS GS SDjjrGS SEjr\GRGS SFj5rGS SGjrGSSHjrGSSIjr\"GR&\"GR(\"GR*\"GR(0\"GR,\"GR.\"GR0\"GR2\"GR4\"GR6\"GR8\"GR:\"GR<\"GR>\"GR@4 Vs0sHnX_M snErSJ\SK'GSSLjrGSSNjr"SOSP5r"SQSR\/5r."SSST5r\ GRL"SU\ GRNSV9rGSSWjr"SXSY\=\\N\5r\R"SZS[55r\"GS0S\\"\%GRZS]S^S_9_S`\"\%GRZSaSbScSd9_Se\"\%GRZSfSgShSd9_Si\"\%GRZSjSkSlSd9_Sm\"\%GRZSnSoSpSd9_Sq\"\%GRZSrSsSqSt9_Su\"\%GRZSvSwSxSd9_Sy\"\%GRZSzS{S|SyS}9_S~\"\%GRZSS~S_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSSSS9_S\"\%GRZSSSSt9_S\"\%GRZSSSSd9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSSSd9_S\"\%GRZSSSSd9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_S\"\%GRZSSS_9_6rS\S'GSSjr"SS\A5r"SS\D5r"SS\5r\R"SS55r"SS5r\"5r"SS5r"SS5r\ "S\kS9r\ "SM\\S9r\(a\\"GRv\P\\\\S444r"SS\\\45r"SS5r"SS\\\5r\R"SS55r\GRGSSj5r"SGS5r"GSGS\>5rgs snf() annotationsN)ABCabstractmethod)autoEnum)chain) AnyCallablecastClassVarGeneric NamedTupleOptional TYPE_CHECKINGUnion)SelfTypeVar)ELEMENTWISE_TYPE_PROMOTION_KIND)_pytree) OrderedSet)int_oo) PythonPrinter)free_symbol_is_typesymbol_is_typeSymT) bound_sympy ValueRanges)configmetrics)DtypePropagationOpsHandler)BasicMathOpsMixinDefaultHandler) boolean_opsDeferredLineBasegenerate_assertget_current_backendIndentedBuffer ir_dataclass ScopedDict sympy_dotsympy_index_symbol sympy_subs triton_typeunique)ops OpsHandlerOpsValue ReductionType StoreModeV)IteratorMutableMappingSequence) GraphModule)CustomGraphModulePass)Buffer ChoiceCaller FixedLayoutIRNodeLoopBody)BaseScheduling Scheduler SchedulerNodePythonWrapperCodegen_Tschedulec[R[R5(a[R SU5 gg)NzData type propagation: %s) schedule_log isEnabledForloggingDEBUGdebug)msgs X/var/www/fran/franai/venv/lib/python3.13/site-packages/torch/_inductor/codegen/common.pydata_type_loggerrQWs-  //6<0cj\rSrSr%SrS\S'S\S'S Sjr\SSj5rSS jr \SS j5r S r g )FileBackedGraphModule\z Output of FX wrapper codegen. Exposes the same methods as ModuleType, but these map back to a GraphModule instead of Python source. r9gmzCallable[..., Any] compiled_fnc.[R"SSSS9Ul[R"[R URR 5 URnURUR5 SSS5 g!,(df  g=f)Nzw+z.pyF)modesuffixdelete) tempfileNamedTemporaryFileatexitregisterosremovenamewritevalue)selffs rP __post_init__#FileBackedGraphModule.__post_init__fs^!33eE    4==#5#56 ]]a GGDJJ ]]s !B Bc.URR$N)r\rbres rP__file__FileBackedGraphModule.__file__ps}}!!!rRc UR"U6$rj)rWreargss rPcallFileBackedGraphModule.callts&&rRc.URR$rj)rVcoderks rPrdFileBackedGraphModule.valuewsww||rR)r\NreturnNonerwstr)rp list[Any]rwr ) __name__ __module__ __qualname____firstlineno____doc____annotations__rgpropertyrlrqrd__static_attributes__rRrPrTrT\sF O## ""'rRrTcH\rSrSrSrSrSr\S Sj5r\S Sj5r Sr g) WorkspaceZeroMode|rrDrcX:XdU[R:XaU$U[R:XaU$[SU<SU<S35e)NzWorkspaceZeroMode.combine(, ))r UNINITIALIZEDNotImplementedErrorabs rPcombineWorkspaceZeroMode.combinesI 6Q+999H !// /H!$>qe2aU!"LMMrRcPU(a[R$[R$rj)r ZERO_ON_CALLr) zero_fills rP from_boolWorkspaceZeroMode.from_bools $11 1 ...rRrN)rrrrrwr)rboolrwr) r|r}r~rrrZERO_PER_GRAPH staticmethodrrrrrRrPrr|s9MLNNN//rRrc@\rSrSrSr\SSj5r\SSj5rSrg) CodegenSymbolzH An IR object possibly corresponding to a variable in the wrapper code. cgrjrrks rPget_nameCodegenSymbol.get_name rRcgrjrrks rP get_exampleCodegenSymbol.get_examplerrRrNryrwz!Union[torch.Tensor, sympy.Symbol]) r|r}r~rrrrrrrrRrPrrs/    rRrT)frozencD\rSrSr%SrS\S'S\S'S\S'S \S 'S rS \S '\Rr S \S'\ SS Sjj5r \ S!Sj5r \ S"Sj5r \ S"Sj5rS#Sjr\rS$SjrS%SjrS&Sjr\S&Sj5r\r\r\rS'SjrS(SjrS(SjrS)SjrS*SjrSrg)+ WorkspaceArgaA temporary buffer used for a single kernel, then discarded. Not registered as a traditional buffer since there are no users, so it would be dead code eliminated. Args: nbytes: The size of the buffer in bytes. zero_fill: Whether the buffer should be initialized to zero. sympy.Exprcountr zero_mode torch.devicedevicerz outer_namews_ptr inner_name torch.dtypedtypecPU[[RR53$rj)nextr5graph workspace_id)prefixs rP unique_nameWorkspaceArg.unique_names!$qww334566rRcURUR:H=(a9 URUR:H=(a URUR:H$rj)rrrrs rPcan_joinWorkspaceArg.can_joins@ LLALL ( XQWW-? XAHHPQPXPXDX rRc [URUR-[RURUR5UR UR URURS9$N)rrrrrr) rrrrrrrrrrs rPjoinWorkspaceArg.joinsS''AGG#'// Q[[I''88||||   rRc URUR:Xa4URUR:XaURUR:Xde[[R "UR UR 5[RURUR5URURURURS9$r) rrrrsympyMaxrrrrrrs rPmaximumWorkspaceArg.maximums GGqww 188qxx#7ALLALLUR5R5$rj) get_layoutrrks rPrWorkspaceArg.get_examples ,,..rRc`SSKJn U"URURUR/S/S9$)Nr)r=rD)rrsizestride)irr=rrr)rer=s rPrWorkspaceArg.get_layouts.$;;****3   rRc"UR5$rj)rrks rPlayoutWorkspaceArg.layouts  rRc6[RR$rj)rSZerorks rP get_offsetWorkspaceArg.get_offsetsww||rRcUR/$rj)rrks rPget_sizeWorkspaceArg.get_sizes |rRc8[RR/$rj)rrOnerks rP get_strideWorkspaceArg.get_strides }rRcUR$rj)rrks rPrWorkspaceArg.get_names rRc/$rjrrks rPget_inputs_that_alias_output)WorkspaceArg.get_inputs_that_alias_outputs rRrN) workspace_)rrzrwrz)rrrrrwr)rrrrrwr)rwr)rwrr)rwr=)rwr)rwlist[sympy.Expr]ry)rw list[str])r|r}r~rrrrtorchuint8rrrrrrrget_device_or_errorrrrrrget_output_specmaybe_get_output_specmaybe_get_layoutrrrrrrrrRrPrrs    OJE;$77        %/ !!!O&!rRrc(\rSrSrSSjrSSjrSrg)TritonScratchWorkspaceicXlX lgrj)r_generate_dtype_str)rergenerate_dtype_strs rP__init__TritonScratchWorkspace.__init__s  #5 rRc"UR5$rj)rrks rPr)TritonScratchWorkspace.generate_dtype_strs''))rR)rrN)rintrCallable[..., str]ry)r|r}r~rrrrrrRrPrrs 6*rRrcx\rSrSr%S\S'S\S'S\S'\R RrS\S'S r S \S 'S r g ) TensorArgi rzrbbufferrrroffsetN Optional[str]alias_ofr) r|r}r~rrrrrrrrrrRrPrr s. I K FJ%"Hm"rRrc>\rSrSr%S\S'S\S'\S Sj5rSrg) SizeArgirzrbrexprcgrjrrks rPrSizeArg.alias_ofsrRrNrwr)r|r}r~rrrrrrrRrPrrs I  rRrc \rSrSr%S\S'Srg) ConstexprArgirzrbrNr|r}r~rrrrrRrPr r s IrRr c>\rSrSr%S\S'S\S'S\S'S\S'S rg ) TMADescriptorArgi$rzrbapi_typezOptional[list[sympy.Expr]] block_shapeOptional[torch.dtype]rrNr rrRrPrr$s IM++  rRrc8\rSrSr%S\S'S\S'SrS\S'S rg) DeviceCodegeni,SchedulingConstructor schedulingWrapperConstructorwrapper_codegenNOptional[WrapperConstructor]cpp_wrapper_codegenr)r|r}r~rrrrrrRrPrr,s%%''8<5rArDrGrJrMrPrUrrrRrPrr8so""""""""""""""""""#9" ("rRrzdict[str, DeviceOpOverrides]device_op_overrides_dictz*dict[str, Optional[CustomGraphModulePass]]custom_backend_passesc<[XU5[U'U[U'grj)rrrY)rdevice_schedulingdevice_wrapper_codegendevice_cpp_wrapper_codegendevice_custom_passs rPregister_backend_for_devicer_s&,3MOF%7&!rRc\rSrSr\"5r\"5r\"5r\"5r\"5r \"5r \"5r \"5r \"5r \"5rSrg)BackendFeatureirN)r|r}r~rrFOREACH BUCKETIZEINPLACE_BUFFERSMASKED_SCATTER_WITH_INDEXSCANSORTTUPLE_REDUCTIONPREFER_STORE_LOOP_ORDERTRITON_TEMPLATESREDUCE_TO_SINGLE_ELEMENTrrrRrPrarasLfGIfO $ 6D 6DfO"fv#vrRrac\Uc [5$[5 [U[R5(a UR nO=[U[ 5(d[ U55eUn[R"U5n[U5nU(deU"S5nURU5$rj) rinit_backend_registration isinstancerrtyperzget_scheduling_for_deviceget_backend_features)r device_typescheduling_ctorrs rPrqrqs~|&%,,''kk &#&&4V 4& k*/ T[R"U5$rj)r cpu_backend)r cpu_backendss rP+init_backend_registration..s|F,>,>? KrRcuda)rrc6>T[R"U5$rj)r cuda_backend)r cuda_backendss rPrrs}V-@-@A*MrRxpumps privateuseoner)_get_custom_mod_func SchedulingrFCppWrapperCodegen)rrcpp_wrapper_cpurcpp_wrapper_cpu_array_refrcpp_wrapper_gpurcpp_wrapper_mpsrcuda_combined_schedulingrrrrrrrwrapperrFrpr_r aot_inductorallow_stack_allocationr_C_get_privateuse1_backend_name torch.utils.backend_registrationr RuntimeError)rrrrrrrrrrFprivate_backendrr[rrrrs @@rPrmrmsp".@..@($(- '/ && $  K ""99 "  !(0-&  $  M   !'/#     !'/#     hh<<>O?* %o 6 >I  4\ B 23IJO"67J"K  _9L+#%#' :M_  ? +    s;E E E cJSSKJn /UQ[XRU55P$)Nr)FlexibleLayout)rrr+contiguous_strides)index index_varssizesrs rPindex_prevent_reorderingr!s* $ UU TIj*K*KE*RS TTrRcU[U'grj)rX)rdevice_op_overridess rPregister_device_op_overridesr,s(;V$rRc[U[5(d[U55e[(dSSKJnJn SSKJn SSK Jn [U$)NrD)cpu_device_op_overridesmps_device_op_overrides)r) rnrzrorXrrrrr)rrrrxpu_op_overridess rPget_device_op_overridesr2s8 fc " "0DL0 " # #F-@ #F ++rRzdict[torch.dtype, torch.dtype]DTYPE_TO_COMPUTATION_DTYPEcU[5;a[R$US;aSU;aUS$US$US;a[R$US;a[R$US:XaSU;aUS$US$US:XaSU;aUS$US$US ;a$USn[ R RU5$US :XaSU;aUS$US $g ) zC Given op name and a list of input dtypes, deduce the output dtype )to_dtype index_exprr)randrandn) get_index randint64 load_seed reductionrDconstant)loadstorestore_reductionto_dtype_bitcastN)r$rrfloatint64r5rr)op_namerpkwargsbuf_names rPdeduce_output_dtype_by_namerSs +-zz  #*V"3vgAbA  {{   {{ K ")V"3vg@a@ J ")V"3vgAbA   7ww  ** & &")V"3vgAbA rRCSEVariableTypec*[5n[RR(a(US:Xa"UR SUS[ U5S35 g[RR (aUS:XaSSKJnJ n [X5(d[U55eU[R:Xa"UR(aSUS 3nO2S US US 3nO(S US3nUR(aSUS3nSUSXRS3nUR SUS35 ggg)Nrztl.static_assert(z .dtype == rrrD)CppCSEVariable DTYPE_TO_CPPzIsVecMaskType::valuezstd::is_same_v || std::is_same_vz decltype(z typename z ::value_typezstd::is_same_vzstatic_assert(z);)r'r test_configsruntime_triton_dtype_assert writeliner.static_cpp_dtype_assert cpp_utilsrrrnrorris_vec)rvarrbackendrr is_same_dt c_var_types rP check_dtyper}s"#G 667h;N,SEK*R89!:J 4rRcx\rSrSrS SjrS SjrSSjrS SjrSSjrSSjr \ SSj5r \ SS j5r S r g )DataTypePropagationicXlSURR0UlURR 5Hup#URURU'M gNroot)body root_blockrgraphs subblocksitems)rerkvs rPrDataTypePropagation.__init__sL DOO))B  NN((*DAWWDKKN+rRcURnUVs/sHBn[U[RR5(dM.UR S:wdM@UPMD nn[ U5S:Xag[SU55nU(dg[R"[RUVs/sH)o3R[RRPM+ sn5$s snfs snf)N placeholderrc3# UHRn[RUR;=(a) UR[RRSLv MT g7frj)OptimizationContextkeymetar).0ns rP BDataTypePropagation.deduce_node_dtype_by_inputs..sR) ! # #qvv - B*../55TA B sAA)all_input_nodesrnrfxNodeoplenall functoolsreduce promote_typesrrrr)renodeinputsr input_nodesall_input_nodes_propagateds rPdeduce_node_dtype_by_inputs/DataTypePropagation.deduce_node_dtype_by_inputss%% !Auxx}}!=A!$$-BWAv   { q %() !) & " *   T$++->>+ ;; ! !"3 4 455d; ;8 ++ L    //55rRcUR(deSnURHn[RUR;aUR[RnO [5nUR U5UlXCR[R'UR S:XdMUR nM U$)Nr )nodesrrrrrr)rer graph_dtyperopt_ctxs rPr#DataTypePropagation.propagate_graphs{{{-1 KKD"&&$))3))$7$;$;<-/ 2248GM18II)-- .{{h&%mm  rRc>URURS5$r)rrrks rP propagateDataTypePropagation.propagates##DKK$788rRc.U"U5R5$rj)r)clsrs rPpropagate_loopbody&DataTypePropagation.propagate_loopbodys4y""$$rRcSSKJn SSKJn [ X5(d[ U55e[ UR U5(d[ UR 55e[RUR 5$)Nrr?)rC) loop_bodyr@ schedulerrCrnro_bodyrr)rrr@rCs rPpropagate_scheduler_node,DataTypePropagation.propagate_scheduler_nodes\(-$..:T :.$**h//Adjj1AA/"55djjAArR)rrN)rr@rwrx)r torch.fx.Noderwr)rr'rwr)rztorch.fx.Graphrwr)rwr)rr@rwr)rrCrwr)r|r}r~rrrr rrr classmethodrr%rrrRrPrrsJ% * 6:$9%%BBrRrcD^\rSrSrSSS.SU4SjjjrSrU=r$)riT)simplifypc>U(ag[U[R5(aH[[R S5(a)[R R RU5n[TU]%U5$)Nsizevars) rnrExprhasattrr5rr-r*superdoprint)rerr*r+ __class__s rPr1PythonPrinter.doprintsQ  444*9U9U77##,,T2Dwt$$rRr)rrr*rr+rrwrz)r|r}r~rr1r __classcell__r2s@rPrrs248D%%-1%=A% %%rRrc\rSrSrSr\SSj5r\SSj5r\SSj5r\SSj5r \SSj5r \SSj5r \SS j5r \SS j5r \SS j5r\SS j5r\SS j5r\SSj5r\SSj5r\SSj5r\SSj5r\SSj5r\SSj5r\SSj5rSrg)OpDecompositionsiz Decomposes inductor ops cU$rjr)rds rPidentityOpDecompositions.identity s  rRcv[R"[R"S[R5U5$NrD)r0truedivrrint32xs rP reciprocalOpDecompositions.reciprocals"{{3<<5;;7;;rRc.[R"X5$rj)r0mulr?s rPsquareOpDecompositions.squareswwq}rRc[R"[R"S[R5[R "U55$r<)r0subrrfloat32erfr?s rPerfcOpDecompositions.erfcs*wws||Au}}5swwqzBBrRc[R"[R"[R"U55[R"U55$rj)r0rDexprErKr?s rPerfcxOpDecompositions.erfcxs,wwswwszz!}-sxx{;;rRc[R"[R"U5[R"S[R 55$r<)r0rHrNrrrIr?s rPexpm1OpDecompositions.expm1s*wwswwqz3<<5==#ABBrRc [R"[R"U5[R"S[R"S5- [ R 55$)NrD r0rDlogrmathrrIr?s rPlog10OpDecompositions.log10"s7wwswwqz3<<DHHRL0@%--#PQQrRc [R"[R"U5[R"S[R"S5- [ R 55$)NrDrrVr?s rPlog2OpDecompositions.log2&s6wwswwqz3<<DHHQK#OPPrRc [R"[R"U[R"[R "S5[ R555$)Nr)r0rNrDrrXrWrrIr?s rPexp2OpDecompositions.exp2*s3wwswwq#,,txx{EMM"JKLLrRc [R"[R"U[R"S[R 555$r<)r0rWaddrrr>r?s rPlog1pOpDecompositions.log1p.s+wwswwq#,,q%++">?@@rRc [R"S[R5n[R"U[R "U[R "[R"U5555$r<)r0rrr>r=rbrNneg)r@ones rPsigmoidOpDecompositions.sigmoid2sCll1ekk*{{3SWWSWWQZ-@ ABBrRcv[R"U[R"S[R55$Nr)r0rrrr>r?s rPreluOpDecompositions.relu7s"{{1cll1ekk:;;rRcX[R"[R"X5U5$rj)r0rbrDr@yzs rPfmaOpDecompositions.fma;swwswwq}a((rRcX[R"[R"U5U5$rj)r0rfloorrrs rP floor_to_intOpDecompositions.floor_to_int@||CIIaL%00rRcX[R"[R"U5U5$rj)r0rceilrvs rP ceil_to_intOpDecompositions.ceil_to_intDs||CHHQK//rRcX[R"[R"U5U5$rj)r0rtruncrvs rP trunc_to_intOpDecompositions.trunc_to_intHryrRc [R"X5n[R"[R"U[R"S[ R 55[R"[R"U5[R"U555n[R"U[R"X!5U5$rk) r0modand_nerrr>signbitwhererb)rrrconds rP remainderOpDecompositions.remainderLsu GGAMxx FF1cll1ekk2 3 FF3;;q>3;;q> 2 yyswwq}a00rRcX[R"[R"U5U5$rj)r0rroundrvs rP round_to_intOpDecompositions.round_to_intUryrRrN)rdOpVarTrwrr@rrwr)r@rrprrqrrwr)rrrrrwrrrrrrwr)r|r}r~rrrr9rArErKrOrRrYr\r_rcrhrlrrrwr|rrrrrrRrPr7r7s}<<CC<<CCRRQQMMAACC<<))1100111111rRr7z[a-z0-9_.]+|\([^)]*\)|)flagscUSS:wd[U5S:agSn[USS5H8up#US:XaUS- nO US:XaUS-nUS:XdM$U[U5S- :wdM8 g US:Xdeg)Nr(rFrDrT)r enumerate)stringrichars rP_all_in_parensr]s ayC3v;? EVABZ( 3; QJE S[ QJE A:!s6{Q. ) A:: rRc\rSrSr\S"Sj5r\S#Sj5r\S$Sj5r\S%Sj5r\S&Sj5r \S&Sj5r \S&Sj5r \S&S j5r \S&S j5r \S'S j5r\S(S j5rS)S*SjjrS+SjrS,SjrS-S.SjjrS/SjrS0SjrS1SjrS2SjrS3S4SjjrS5SjrS\R6S SS.S6SjjrS7SjrS8Sjr\S9Sj5r\ S:Sj5r!\ S;S j5r"S!r#g)< OpOverridesilc[U[5(d*[RU5(d[ U5(aU$SUS3$Nrr)rn CSEVariable_RE_PAREN_NOT_NEEDED fullmatchr)rs rPparenOpOverrides.parenmsB v{ + +#--f55f%%M6(!}rRc[U5$rj)repr)rdrs rPrOpOverrides.constantxs E{rRc2S[RU53$)N~rrr?s rP bitwise_notOpOverrides.bitwise_not|s;$$Q'())rRc2[RU5S3$)Nz == 0r)rs rP logical_notOpOverrides.logical_nots##A&'u--rRc\[RU5S[RU53$)Nz & rr@rps rP bitwise_andOpOverrides.bitwise_and+##A&'s;+<+> rrs rPbitwise_right_shiftOpOverrides.bitwise_right_shiftrrRc.[R"X5$rj)r0r=rs rP int_truedivOpOverrides.int_truedivs {{1  rRcX[R"U[R"U55$rj)r0rrInteger)rbrs rPrOpOverrides.load_seedsxxemmF344rRTc*[[U55$rj)r,rz)rerrcheckwrap_negs rPindirect_indexingOpOverrides.indirect_indexings"#c(++rRcD[[U5RS35e)Nz,: check_bounds should be handled by CSEProxyrror|rerrloweruppers rP check_boundsOpOverrides.check_boundss'"Dz""##O P  rRcD[[U5RS35e)Nz$: load should be handled by CSEProxyrrerbrs rPrOpOverrides.loads%!Dz""##G H  rRNcD[[U5RS35e)Nz%: store should be handled by CSEProxyrrerbrrdrYs rPrOpOverrides.stores'"Dz""##H I  rRcD[[U5RS35e)Nz/: store_reduction should be handled by CSEProxyrrerbrrds rPrOpOverrides.store_reductions%!Dz""##R S  rRcD[[U5RS35e)Nz): reduction should be handled by CSEProxyrrer src_dtypereduction_typerds rPrOpOverrides.reductions'"Dz""##L M  rRcD[[U5RS35e)Nz$: scan should be handled by CSEProxyrredtypes combine_fnvaluess rPscanOpOverrides.scans'"Dz""##G H  rRcD[[U5RS35e)Nz$: sort should be handled by CSEProxyrrerrstable descendings rPsortOpOverrides.sorts'"Dz""##G H  rRcD[[U5RS35e)Nz): bucketize should be handled by CSEProxyrrer boundariesboundary_indicesindexing_dtyperightsortersorter_indicess rP bucketizeOpOverrides.bucketizes'"Dz""##L M  rRcD[[U5RS35e)Nz2: halide_clamp only implemented for Halide backendr)rerdrrs rP halide_clampOpOverrides.halide_clamps%!Dz""##U V  rRrD) constraintsris_purepackcD[[U5RS35e)Nz<: inline_asm_elementwise only implemented for Triton backendr)reasmrrrrrs rPinline_asm_elementwise"OpOverrides.inline_asm_elementwises'"Dz""##_ `  rRcD[[U5RS35e)Nz.: ops.output should not appear at codegen timeAssertionErrorror|ros rPr OpOverrides.outputs%Dz""##Q R  rRcD[[U5RS35e)Nz3: ops.placeholder should not appear at codegen timerrers rPrOpOverrides.placeholder s%Dz""##V W  rRc4^SU4SjjnTUlSUlU$)NcJ>[[U5RST35e)Nz does not implement ops.r)rerprrbs rP unimplemented1OpOverrides._unimplemented..unimplementeds*%:&&''?vF rRT)rerrpr rr rwr)r|is_unimplemented)rbr s` rP_unimplementedOpOverrides._unimplementeds   "& )- &rRc[XS5n[[US5nU(+=(d X#:H=(d [USS5$)Nr F)getattrr1)rrbfn default_fns rP_is_unimplementedOpOverrides._is_unimplementeds; S %Zt4 vS)SWR9KU-SSrRc`US;dU5e[R5Hup#[X15nUc5URU5(a[ XUR U55 MFMHX R ;dSUSUR35eX$l[ X[U55 M g)N)rrcppvecrrzmultiple definitions of z on ) pointwise_overrides_datarrrsetattrr__dict__r|r)rrfuncnamedataimpls rP_initialize_pointwise_overrides+OpOverrides._initialize_pointwise_overrides"sEEMvME6<<>NH4(D|((22C3+=+=h+GH3 ||3.xjS\\NK3!) |D'9:?rRr)rrrwr)rdzUnion[bool, float, int]rrrwrr)rrrwr)r@rrprrwrr)rbrzrrrwrTT) rrrUnion[sympy.Expr, int]rrrrrw sympy.Symbol rrrrrrrrrwrx)rbrzrrrwrrj) rbrzrrrdrrYr4rwrx)rbrzrrrdrrwrx) rrrrrr3rd!Union[OpVarT, tuple[OpVarT, ...]]rwr$)rtuple[torch.dtype, ...]rzFCallable[[tuple[OpVarT, ...], tuple[OpVarT, ...]], tuple[OpVarT, ...]]rtuple[OpVarT, ...]rwr&) rr%rr&rrrrrwr&NN)rrr.tuple[str, sympy.Expr, sympy.Expr, sympy.Expr]rrrrrrr Optional[tuple[str, sympy.Expr]]rzOptional[OpVarT]rwr)rdrrrrrrwr)rrrrzrrrrrrrrrwr)rprrwrx)rrrwr)rbrzrwzCallable[..., OpVarT]rbrzrwr)rrzrwrx)$r|r}r~rrrrrrrrrrrrrrrrrrrrrrrrrIrr rrr(rrrrrRrPrrls'**..BBBBBBCCCC!! 55 , ,%, ,  ,  ,  &0 9= FJ   NR  * 39 AJ        &  1  +    '     #       '  #        $48+/    C  !  $    1  )      &*"]]      #             TT  ; ;rRrc\rSrSr%S\S'S\S'SrS\S'SrS\S '\Rr S \S 'Sr S\S 'Sr S\S 'Sr g) OverridesDatai3rzrbrrNzOptional[Callable[..., str]]rrrtype_promotion_kindrrr) r|r}r~rrrrrDEFAULTr-rrrrrRrPr,r,3sQ I +/F (/+/F (/'//8,0F (/(,C %,rRr,airy_aicSUS3$)Nzairy_ai_forward(rrr?s rPrrGs(1-rRspecial_airy_ai)r-rrb bessel_j0cSUS3$)Nzbessel_j0_forward(rrr?s rPrrL*1#Q/rRcSUS3$)Nz libdevice.j0(rrr?s rPrrM =1-rRspecial_bessel_j0)r-rrrb bessel_j1cSUS3$)Nzbessel_j1_forward(rrr?s rPrrRr4rRcSUS3$)Nz libdevice.j1(rrr?s rPrrSr6rRspecial_bessel_j1 bessel_y0cSUS3$)Nzbessel_y0_forward(rrr?s rPrrXr4rRcSUS3$)Nz libdevice.y0(rrr?s rPrrYr6rRspecial_bessel_y0 bessel_y1cSUS3$)Nzbessel_y1_forward(rrr?s rPrr^r4rRcSUS3$)Nz libdevice.y1(rrr?s rPrr_r6rRspecial_bessel_y1digammacSUS3$)Nz calc_digamma(rrr?s rPrrds aS*rRc US3$)Nz .digamma()rr?s rPrres A3j)rR)r-rrrbrOcSUS3$)Nz calc_erfcx(rrr?s rPrrl A3a(rRcSUS3$)Nzlibdevice.erfcx(rrr?s rPrrms+A3a0rR special_erfcxrrcSUSUSUS3$)Nz std::fma(rrrros rPrrrsis"QCr!A6rRcSUSUSUS3$)Nzfmadd(rrrros rPrrsss"QCr!A6rRcSUSUSUS3$)Nzlibdevice.fma(rrrros rPrrtss"QCr!A>rR)r-rrrrbigammacSUSUS3$Nz calc_igamma(rrrrs rPrrz<s"QCq1rRigammaccSUSUS3$Nz calc_igammac(rrrrs rPrr=2aS2rRgammainccSUSUS3$rPrrs rPrrrQrRspecial_gammainc gammaincccSUSUS3$rTrrs rPrrrUrRspecial_gammaincci0cSUS3$)Nzcalc_i0(rrr?s rPrr 1orRcSUS3$Nzlibdevice.cyl_bessel_i0(rrr?s rPrr3A3a8rRc US3$)Nz.i0()rr?s rPrrs A3erR)r-rrrrbi0ecSUS3$)Nz calc_i0e(rrr?s rPrr !A&rRc US3$)Nz.i0e()rr?s rPrrs A3frR special_i0ei1cSUS3$)Nzcalc_i1(rrr?s rPrrr^rRcSUS3$Nzlibdevice.cyl_bessel_i1(rrr?s rPrrrarR special_i1i1ecSUS3$)Nz calc_i1e(rrr?s rPrrrerR special_i1elog_ndtrcSUS3$)Nzcalc_log_ndtr(rrr?s rPrrs qc+rRspecial_log_ndtrmodified_bessel_i0cSUS3$)Nzmodified_bessel_i0_forward(rrr?s rPrr3A3a8rRcSUS3$r`rr?s rPrrrarRspecial_modified_bessel_i0modified_bessel_i1cSUS3$)Nzmodified_bessel_i1_forward(rrr?s rPrrrurRcSUS3$rkrr?s rPrrrarRspecial_modified_bessel_i1modified_bessel_k0cSUS3$)Nzmodified_bessel_k0_forward(rrr?s rPrrrurRspecial_modified_bessel_k0modified_bessel_k1cSUS3$)Nzmodified_bessel_k1_forward(rrr?s rPrrrurRspecial_modified_bessel_k1ndtrcSUS3$)Nz calc_ndtr(rrr?s rPrrs 1#Q'rR special_ndtrndtricSUS3$)Nz calc_ndtri(rrr?s rPrrrHrR special_ndtri polygammac *USUSUSUSUSUS3 $)Nz == 0 ? calc_digamma(z) : (z == 1 ? trigamma(z) : calc_polygamma(rz))rrs rPrrs3S%aSaS0A!DWXYWZZ\]^\__a brRscaled_modified_bessel_k0cSUS3$)Nz"scaled_modified_bessel_k0_forward(rrr?s rPrr:1#Q?rR!special_scaled_modified_bessel_k0scaled_modified_bessel_k1cSUS3$)Nz"scaled_modified_bessel_k1_forward(rrr?s rPrrrrR!special_scaled_modified_bessel_k1spherical_bessel_j0cSUS3$)Nzspherical_bessel_j0_forward(rrr?s rPrrs4QCq9rRspecial_spherical_bessel_j0zetacSUSUS3$)Nzzeta(rrrrs rPrrs52aS*rR special_zetachebyshev_polynomial_tcSUSUS3$)Nzchebyshev_polynomial_t_forward(rrrrs rPrr:1#Rs!DrRspecial_chebyshev_polynomial_tchebyshev_polynomial_ucSUSUS3$)Nzchebyshev_polynomial_u_forward(rrrrs rPrrrrRspecial_chebyshev_polynomial_uchebyshev_polynomial_vcSUSUS3$)Nzchebyshev_polynomial_v_forward(rrrrs rPrrrrRspecial_chebyshev_polynomial_vchebyshev_polynomial_wcSUSUS3$)Nzchebyshev_polynomial_w_forward(rrrrs rPrrrrRspecial_chebyshev_polynomial_wlegendre_polynomial_pcSUSUS3$)Nzlegendre_polynomial_p_forward(rrrrs rPrr9!BqcCrRspecial_legendre_polynomial_pshifted_chebyshev_polynomial_tcSUSUS3$)Nz'shifted_chebyshev_polynomial_t_forward(rrrrs rPrrB1#Rs!LrR&special_shifted_chebyshev_polynomial_tshifted_chebyshev_polynomial_ucSUSUS3$)Nz'shifted_chebyshev_polynomial_u_forward(rrrrs rPrrrrR&special_shifted_chebyshev_polynomial_ushifted_chebyshev_polynomial_vcSUSUS3$)Nz'shifted_chebyshev_polynomial_v_forward(rrrrs rPrr rrR&special_shifted_chebyshev_polynomial_vshifted_chebyshev_polynomial_wcSUSUS3$)Nz'shifted_chebyshev_polynomial_w_forward(rrrrs rPrrrrR&special_shifted_chebyshev_polynomial_whermite_polynomial_hcSUSUS3$)Nzhermite_polynomial_h_forward(rrrrs rPrrs82aSBrRspecial_hermite_polynomial_hhermite_polynomial_hecSUSUS3$)Nzhermite_polynomial_he_forward(rrrrs rPrrrrRspecial_hermite_polynomial_helaguerre_polynomial_lcSUSUS3$)Nzlaguerre_polynomial_l_forward(rrrrs rPrr!rrRspecial_laguerre_polynomial_lzdict[str, OverridesData]rc^[U4Sj[RR[RR[RR [RR 455$)Nc3.># UH nTU;v M g7frjr)rr@rbs rPr$is_buffer_removed..(s  A   s)anyr5rremoved_bufferskernelinplaced_to_removerbs`rPis_buffer_removedr'sU  GG # # HH $ $ GG & & HH ' '   rRcD^\rSrSrSrSU4SjjrSSjrS SjrSrU=r $) DeferredLinei3zHA line that can be 'unwritten' by adding name to V.graph.removed_buffersc^>[TU]U5 Xl[U[5(aegrj)r0rrbrnr%)rerbliner2s rPrDeferredLine.__init__6s,  d$455555rRcP[UR5(d UR$grj)rrbrrks rP__call__DeferredLine.__call__;s ++99 rRc.[URU5$rj)rrb)rers rP _new_lineDeferredLine._new_line@sDIIt,,rRr)rbrzrrzr )rrzrwr) r|r}r~rrrrrrr4r5s@rPrr3sR6  --rRrc"\rSrSrSSSjjrSrg) BracesBufferiDcL^^[RSUU4Sjj5nU"5$)Nc3># [T5H)nTRS5 T=RS- slM+ [T*5H)nT=RS-slTRS5 M+ Sv [T*5H)nTRS5 T=RS- slM+ [T5H)nT=RS-slTRS5 M+ g7f)N{rD})ranger_indent)_rres rPctx BracesBuffer.indent..ctxFs6]s# ! #F7^ ! s#$ F7^s# ! $6] ! s##sC(C+)rwIterator[None]) contextlibcontextmanager)rerrs`` rPindentBracesBuffer.indentEs$  " " $ # $u rRrN)rD)rrrwz'contextlib.AbstractContextManager[None])r|r}r~rrrrrRrPrrDs rRrc*\rSrSr%S\S'S\S'Srg)InplacedBufferiYrzrr other_namesrNr rrRrPrrYs OrRrc8\rSrSr%S\S'SrS\S'S SjrSrg ) ArgNamei^rzrbFr is_constexprcLURUR(aS3$S3$)Nz : tl.constexprr)rbrrks rP full_nameArgName.full_nameds*))$2C2C.LMMLMMrRrNry)r|r}r~rrrrrrrRrPrr^s IL$NrRrc\rSrSrSSjrSrg) RemovedArgihcg)NREMOVEDrrks rP__str__RemovedArg.__str__isrRrNry)r|r}r~rrrrrRrPrrhsrRrc\rSrSr\SSj5rSSjrSSjr\SSj5rSSjr SSjr SSjr SS jr SS jr S S jrS!S jrS"S jrS#SjrS$SjrS%SjrS&S'SjjrS(SjrS)SjrS*SjrS+SjrSrg), KernelArgsipcURU[5n[U[5(aU[ U53=X'nU$U$rj)getrrnrr)rodictrbresult new_results rP_lookupKernelArgs._lookupqsE */4)A fj ) )*0#e*'> >EK*  rRcJ0Ul0Ul0Ul0Ul/Ulgrj) input_buffersoutput_buffersinplace_buffersr-workspace_argsrks rPrKernelArgs.__init__}s)-/ACMO/1 24rRc SRSR[[URUR UR UR/555$)NzKernelArgs({})r)formatrmaprrrrr-rks rP__repr__KernelArgs.__repr__sS&& II**++,,    rRc"[U[5$rj)rnrrs rP_buffer_is_marked_removed$KernelArgs._buffer_is_marked_removeds$ ++rRcT[RR(a3[RRRR X5nU[RR ;dU5eXR ;a[[UR U5$XR;a'[[URU5R$URS5(aURSURU5$URSURU5$)Nseedin_ptr)r5rr#mutation_real_namerrrr rzrrrrrrrs rPinputKernelArgs.inputs 77  77$$77;;DGD1772228D82 && &T0067 7 '' '(<(R!R  /01)C03  ,03  -! sF 6F F7FcP[U[RU5[RR 5[R 5S9n[UR5HupE[RXS5(a>URn[RXS5URU'URU4s $URUR:waURUR:waMU5e URRU5 URS4$)a Allocate or extend a workspace buffer of nbytes bytes. This function manages the allocation of a workspace buffer. It either creates a new WorkspaceArg or extends an existing one. Note: - Calling this function will in-place mutate the args by adding or updating a WorkspaceArg. - The codegen for generating the Python argdefs and call_defs will check this field and allocate the buffer accordingly. - A new argument "ws_ptr" will be present in the generated code. Args: nbytes (sympy.Expr): The number of bytes to allocate. zero_fill (bool): Whether to initialize the buffer to zero. Returns: Tuple[str, int]: A tuple containing: - "ws_ptr": A string identifier for the workspace pointer. - offset: An integer representing the byte offset in the workspace. )rrrrr)rrrr5rget_current_device_or_throwrrrrrrrrr)renbytesrargr existing_argrs rPrTKernelArgs.workspaces.'11)<77668#//1   ))<)<=OA$$\77%++)5):):<)M##A&#..66''3>>9 ++s~~=  > > ""3'~~q  rRc [RR5n[U[R [ RSSURSUR3US9nURH,nURUR:XdMX4:XaM&X445e URRU5 UR$)a Lazily allocate a graph-wide semaphores buffer with at least min_size. This is a single buffer shared by all kernels and zero initialized once at graph start. Each kernel must leave the buffer zeroed on exit. Warning: multiple calls to this function will return the same buffer. Args: min_size: the number of int32 semaphores required Returns: name of the semaphores buffer sem_ptr semaphores_r)rrrrrr) r5rrrrrruint32rorrrr)remin_sizecurrent_devicer r!s rP semaphoresKernelArgs.semaphoress<<>'66,, $^%8%8$9>;O;O:PQ!  !//L&&#..8*?S,??*0 ""3'~~rRcz^[U[5(d[U5U45e[R"U5nX R ;aUR U$TUR R 5;a1T[U4SjUR R 5553mTUR U'T$)Nc3V># UHoRT5(dMSv M g7f)rDN)r)rrrbs rPr)KernelArgs.seed_offset..sU(>1,,tBTQQ(>s) ))rnrrorrr-rsum)rerbrds ` rP seed_offsetKernelArgs.seed_offset s%%%;U U';;% e$ MM !==' ' 4=='') )&U (<(<(>UUVW  $ e rRc[U[R5(d[U5U45eURS:XaSUR U'gUR SUR U5$)Nr ks)rnrSymbolrorbr-rrs rPrKernelArgs.sizes[$ --AT D/AA- 99 "(DMM$ ||D$--66rRc[URR5URR5URR55$rj)rrkeysrr-rks rP call_namesKernelArgs.call_names!sA    # # %t':':'?'?'A4==CUCUCW  rRc&URRUS5nUb![U[5(d UR$UR RUS5nUb[U[5(dU$UR RUS5$)z+ Returns inner name of a given outer name. N)rrrnrrrr)rerbinplacedrs rParg_nameKernelArgs.arg_name&s''++D$7   8Z(H(H&& &))--dD9  ":k:+N+N !!%%dD11rRcU$rjr)rerrs rP wrap_ptr_argKernelArgs.wrap_ptr_arg2s rRc[U5$rj)rz)rers rP wrap_size_argKernelArgs.wrap_size_arg5s 4yrRNcSSKJn UcSSKJn Un/n/n/n[URR 55Hn[ U[5(aMURSnURn [RRU5n Xn URU SU 35 URURX55 URU S35 M URR!5HupXR;aM[RRU5n Xn URSU SU 35 URURX55 URSU S35 M UR"R!5HupXR;d[ U [5(aM+[RRU5n Xn URU SU 35 URURX55 URU S35 M UR$R!5HupURSUSU 35 URUR'U55 URSU35 [RR((dMq[RR(R+U5 M UR,(aS 5eXTU4$) NrD) INDEX_TYPE)rrz* *zconst  zWorkspace not supported on CPU )rrDrr/rrrnrrrr5rrrr>rrrr-rA wrapper_codeensure_size_computedr) redtype_to_cpp_typerDr call_argsarg_defs arg_typesr:outerinnerr cpp_dtype maybe_inners rP cpp_argdefsKernelArgs.cpp_argdefs8s *  $ / ,   t33::<=H(J//((,E''EGG%%e,E)0I OOykE73 4   T..u< =    {!_ ->!..446LE,,,GG%%e,E)0I OOfYKr%9 :   T..u< =   vi[2 37#'"5"5";";"= E,,, ; 0S0SGG%%e,E)0I OOykK=9 :   T..u< =    {!_ -#>!MM//1LE OOfZL%9 :   T//6 7   vj\2 3ww###$$99%@ 2 &&I(II&I--rRc /n/n/n/n[URR55Hn[U[5(aMUR [ UR55 UR URS5 UR [RRURS55 UR [URURS[RRURS5S95 M [URR5UR R55HupgX`R;d[U[5(aM+UR [ U55 UR U5 UR [RRU55 UR [UU[RRU5S95 M UR"R5HupgUR [ U55 UR U5 UR [%U55 UR ['Xv55 [RR((dM[RR(R+U5 M UR,HnnUR [ UR55 UR UR.5 UR U5 UR UR05 Mp XXC4$)Nr)rbrr)r/rrrnrrrrrr5rrrrrrrr-rorrGrHrrr) rerKrJrLprecompile_argsr:rMrNr s rPpython_argdefsKernelArgs.python_argdefshs#%! ! /1t33::<=H(J// OOGH$7$78 9   X11"5 6   QWW..x/C/CB/GH I  " "!,,#//3''++H,@,@,DE  >"    $ $ &(;(;(A(A(C LE,,, 5*0M0M OOGEN +   U #   QWW..u5 6  " " ''++E2  !MM//1LE OOGEN +   U #   T%[ )  " "75#8 9ww###$$99%@ 2&&C OOGCNN3 4   S^^ ,  " "3 '   SYY ' ' O>>rRc## [URR55Hn[U[5(aMUR HnU[ RR;dU[ RR;aMAX R;aURUUR4v X R;dM~[[URU5UR4v M M g7frj)r/rrrnrrr5rrrrrrr rz)rer:others rPaliasesKernelArgs.aliasesst33::<=H(J//!--QWW777 ; ;;...,,U3X5H5HHH///sD$7$7$>?ATATTT.>s CC> 4C>c[URRU[5[5=(a. [UR RU[5[5$rj)rnrrrrrrs rP is_removedKernelArgs.is_removedsK    # #D' 2J N--11$@*M NrRc[5n[URR55H8n[ U[ 5(aMUR URS5 M: URR5H<up4X0R;d[ U[ 5(aM+UR U5 M> U$)Nr) rr/rrrnrrbrrr)re live_outsr:rMrNs rPlive_output_buffersKernelArgs.live_output_bufferss%/\ t33::<=H(J// MM(..r2 3>!//557LE,,, 5*0M0M MM% 8rR)rrrr-r)rrzrz6Union[dict[_T, Union[str, RemovedArg]], dict[_T, str]]rbrGrwrzrvry)rbr rwrrW)rrzrrzrwrx)rrrrrwztuple[str, int])r'rrwrz)rbrzrdrrwrz)rbr"rwrz)rwz Iterator[str])rbrzrwr)rrzrrrwrz)r SymbolLikerwrzrj)rIz Optional[dict[torch.dtype, str]]rwz&tuple[list[str], list[str], list[str]])rwz?tuple[list[ArgName], list[str], list[KernelArgType], list[Any]])rwzIterator[tuple[str, str]]r*)rwzOrderedSet[str])r|r}r~rrrrrrr r rrTr)r/rr7r;r>rArQrUrYr\r`rrrRrPrrps  E    5  ,, @B48'!R8 7 2EI..!A.. /..`/? H/?b UN rRrcr^\rSrSrSrS S U4SjjjrS SjrS SjrSSjrSSjr S Sjr S r U=r $)ria4A CSEVariable is just a name for an expression but it is useful to be able to annotate them on a backend dependent basis. To do so, the backends can simply overload `Kernel.create_cse_var` The "CSEVariable.update_on_args" method gives you a hook for annotations See example of TritonCSEVariable in triton.py c>[TU]5 [U[5(d[ U55eXlX lSUlX0lgr<) r0rrnrrorbbounds use_countr)rerbrerr2s rPrCSEVariable.__init__sB &+..<V <.   rRcUR$rjrrks rPrCSEVariable.__str__s yyrRc,[UR5$rj)hashrbrks rP__hash__CSEVariable.__hash__sDIIrRcb[U[5=(a URUR:H$rj)rnrrb)rerXs rP__eq__CSEVariable.__eq__s!%-I%** 2IIrRcgrjr)rerbrprs rPupdate_on_argsCSEVariable.update_on_argss rRcPURRSUR<S3$r)r2r|rbrks rPrCSEVariable.__repr__s$..))*!DII=::rR)rerrbrfrj)rbrzreValueRanges[Any]rrry)rwr)rXobjectrwr)rbrzrpr rr rwrx) r|r}r~rrrrrlrorrrrr4r5s@rPrrsR(,  ! %  J ;;rRr AugmentedKeyT)default)boundry.cf\rSrSrSrSSSjjrSSjrSSjrSSjrSSjr SS jr SS jr SS jr SS jr \R"5S S SS.SSjjr\R"5S4SSjjr\R"5S4SSjjrSrg)CSEiz Common subexpression eliminationNcXlX l0UlX0lU=(d 0UlU=(d 0UlU=(d [ R"5Ul[5Ul U=(d 0Ul grj) rrZ_cache name_prefix store_cachereduction_cache itertoolsriter_buffer_idsrinvalidated_stores varname_map)rerrZr iter_buffersrrrs rPr CSE.__init__sg  FH &ALARPR  !r 6B5VY__EV3=<7B7HbrRcL/URR5QH4up#X1;dM URU URRU5 M6 U(a<URR5VVs0sHupEXQ;dM XE_M snnUlg0Ulgs snnfrj)rrrrbr~)re keep_varsrbtmprrs rP invalidateCSE.invalidates44++1134ID#$$T*''++D15 ,0KK,=,=,?R,?DA1>414,?RDKDKSs 9 B B c [U5"URURURURUR UR URS9$)N)rrZrrrrr)rorrZrrrrrrks rPclone CSE.clonesNDz;;;;((--(((( 00  rRcUR5n[UR5Ul[UR5Ul[UR5UlU$)zNReturn a copy of using ScopedDict so changes to *_cache aren't visible in self)rr*r~rr)renew_cses rP scoped_copyCSE.scoped_copysH**,#DKK0",T-A-A"B()9)9:rRc"[[U5$)z@Override this method to augment cache key with backend specifics)r rxre cache_keys rP augment_keyCSE.augment_key$sM9--rRc>X RURU5'grjr~r)rerrs rPputCSE.put(s36 D$$Y/0rRc>URU5UR;$rj)rr~rs rPcontains CSE.contains+s *dkk99rRcXURRURU5S5$rj)r~rrrs rPtry_get CSE.try_get.s"{{t// :DAArRc>URURU5$rjrrs rPrCSE.get1s{{4++I677rRT)rerc assignmentrc V[U[5(a URnU(d U(de[U[5(aEURR U5UlU=R S- sl[[U5$[U[5(aUR5nO;[U[5(a URnO[U[5(deUnURU5nU(GdURX65nUR!Xx5 U(Ga["R$R&(a(["R$R&R)USS9 [U[5(aUU(a UR+UR,US35 UR/U5 UR+UR05 U$[U[5(aRU(deUR+UR3UR,USURUR0355 U$U(a UR,USUUR03n OUUR03n UR+U 5 U(a[[4R6R8(d[4R6R:(aUb[=5S:wa [?XU5 U$URR U5UlU=R S- slU$)NrDT) only_oncez =z = r) rnr2rdrretightenrfr rr(getvaluer%rrzrnewvarrr5r current_nodecodegen_originating_inforrsplicerZrrrrrr'r) rerrrercrrrrrs rPgenerate CSE.generate4sk dH % %::D "" dK ( (++--f5DK NNa N. . n - - I . / / IdC(( ((Ill9%++f,C HHY $88((HH))BB$CdN33!((DKK=R)@AMM$'$$T[[1: 9 &677%%:$$$++se3tyyk$++'WX4 -""&++se3tfT[[MJ"& }5$$T*#"//KK%22JJ!-/1U:#F7  ++F3CJ MMQ M rRcUR[UR53n[RR X1U5nX@R U'U$rj)rrrr5rcreate_cse_varr)rerervar_namers rPr CSE.newvar|sN &&'T-A-A(B'CDhh%%h>%(" rRc^[R"TUR;U4Sj5 [RR TX#5nX@RT'U$)Nc>ST3$)Nzduplicate name: rrsrPrCSE.namedvar..s 4DTF2KrR)r _check_valuerr5rr)rerbrerrs ` rPnamedvar CSE.namedvarsO  (( (*K hh%%dF:!$ rR) r~rrrrrrrZr)rrrNNNN)rrzrZrzrrzrzOptional[itertools.count[int]]rz.Optional[MutableMapping[str, CSEVariableType]]rz[TU]5 [R"5Ulgrj)r0rr ExitStack exit_stackrer2s rPrCodeGen.__init__s $..0rRc:URR5 U$rj)r __enter__rks rPrCodeGen.__enter__s !!# rRc<URRXU5 grj)r__exit__)reexc_typeexc_valexc_tbs rPrCodeGen.__exit__s   F;rR)rrvrrr rr rr rwrx) r|r}r~rrrrrr4r5s@rPrrs1<[TU]5 U(a[=RS- slU=(d [ 5Ul[ 5Ul[ 5Ul[ 5Ul SUl SUl [URUR5Ul[!5Ul[!5UlSUlSUlSUlSUl[!5Ul[!5Ul0UlSUlSUlg)NrDr)r0rr generated_kernel_countrrpr(loadscomputestoresnum_load num_reductionr|rrZcsermust_keep_buffersstore_buffer_names _load_mask _load_otherrnode_to_boundsrrinplace_update_buffersmin_elem_per_thread kernel_name)rerpincrease_kernel_countr2s rPrKernel.__init__s   * *a / *(JL #% %' $&  .1$2D2Ddkk.R2<,3=<)-4859OS0: 3=< 79##$ *.rRc## URnXlURR5R5UlSv X lg!X lf=f7frj)rr$re get_boundsr)rerpriors rPset_current_nodeKernel.set_current_nodesH!! "jj//1<<> &  %  sAAAAAAc## UcUnUSL=n(a [5nURnURnURnURnXlX lX0lUR 5UlSv XPlX`lXplXlU(aU(aS5egg!XPlX`lXplXlU(aU(aS5eff=f7f)Nz$unexpected store inside swap_buffers)r(rrrrr) relbcbsbdisallow_storesrrrrs rP swap_buffersKernel.swap_bufferss :B Dj (? (!B ,,hh   ??$ F J"L KHEEEv2 J"L KHEEEv2sA2C5B(9/C(0CCc[erjrrs rPr Kernel.loadr"rRcURnURUlURX5X0l$!X0lf=f)z+A load the depends on an index we have read)rrr)rerbrrs rP indirect_loadKernel.indirect_loads2  DJ99T)JJs!6>c[erjrrs rPrKernel.store_reductionr"rRc[erjrrs rPr Kernel.store "!rRc[erjrrs rPrKernel.reduction "!rRc[erjrrs rPr Kernel.scan s "!rRc[erjrrs rPr Kernel.sortrrRc[erjrrks rP var_rangesKernel.var_ranges r"rRc[e)z# See [Note: Inductor bucketize op] rrs rPrKernel.bucketize#s "!rRc[erjrrks rPassert_functionKernel.assert_function2s!!rRc [U[5(a [U5n[U[5(d[U55eUb[U[5(deUb[U[5(deU(a!U(aSUSUSUSUS3 nUSUSU3nO#U(a USU3nUnOU(deUSU3nUnU(a SUSUS3nURSUSUS3$) Nrz <= z) & (z < rz) | ~(z, "index out of bounds: z"))rnrrzror)rerrrmaskr cond_prints rPindirect_assertKernel.indirect_assert6s c; ' 'c(C#s##.T#Y.#} 5# 6 666} 5# 6 666 UugT#eC5E7![TU]5 UR(deURR [ R "[XR5555 URR [ R"U55 U$rj) r0rrr enter_contextr5set_ops_handlerCSEProxyset_kernel_handlerrs rPrKernel.__enter__\sj ~~~ %%  ht^^-=> ?  %%a&:&:4&@A rRcF>UR5 [TU] XU5 grj)remove_kernel_local_buffersr0r)rerrrr2s rPrKernel.__exit__es ((* F3rRc^^[RRmT(dg[U4SjUR55n[5mURHXnX R ;dMX R R;dM/TRX!5(dMGTRU5 MZ THnX R R;aUR RUn[U[5(aML[U4SjUR55nU(aURU5 UR RU5 MUR#U5 M g)z Any buffers that are both created and have a last use in the same kernel can be removed. Note that V.graph.scheduler can be None when codegening triton template kernels. Nc3># UH4nUTR;dMTRUR5v M6 g7frj) name_to_bufdefining_op_name)rrr#s rPr5Kernel.remove_kernel_local_buffers..ts?& .i+++ :I ! !# & 7 7 9 9.s?$?c3,># UH oT;v M g7frjr)rrnames_to_removes rPrrsK?a/1?s)r5rr#rrrrpr$can_buffer_be_removed_through_fusionrbrrnrrrremove_inplace_bufferr remove_buffer)refused_node_namesrbrrar!r#s @@rPr"Kernel.remove_kernel_local_buffersisGG%%  %& ..&   ,6<++D222 7 77BB ##D),$Dyy000ii//5c:..K3??KK..t4''++D1""4($rRc[RSU5 [URRU'UR R U5 g)Nzremove_buffer(%r))rWrNrrprrrbrs rPr$Kernel.remove_buffers; %t,)0   &   &rRc[RSU5 [URRU'UR R U5 g)Nzremoving_inplace_buffer(%r))rWrNrrprrrbrs rPr#Kernel.remove_inplace_buffers9 /6*1 !!$'   &rRc [U[[45(a!UVs/sHo RU5PM sn$[R R RU5n[URSS9nUVs0sH^n[U[R[R[R45(dMCX RR!U5_M` nn[#X5$s snfs snf)NcUR$rjr)ss rPr(Kernel.rename_indexing..s!&&rR)r)rnlisttuplerename_indexingr5rr-r*sorted free_symbolsrr UNBACKED_INTSIZEPRECOMPUTED_SIZErprr-)rerr@sorted_symbols replacementss rPr1Kernel.rename_indexings edE] + +5:;U((+U; ;  ))%0 2 28HI$  #%%II)) !Ayy~~a #   %..<  sC1?AC6C6c[U0UD6$rj)r)rerprs rPrKernel.create_cse_varsD+F++rRc\UcgURRUR55$)z3 Returns arg name of a given input or output node. N)rpr;r)rers rPr;Kernel.arg_names' <yy!!$--/22rR)rrrprrrrrrrrrrrrrrr)NT)rpzOptional[KernelArgs]rrrwrx)rrCrwrr')rr(rOptional[IndentedBuffer]rr>rwrrbrzrrrwrrbrzrrrdrrwrxrj rbrzrrrdrrYr4rwrx rrrrrr3rd+Union[CSEVariable, tuple[CSEVariable, ...]]rwrCrr%rzUCallable[[tuple[CSEVariable, ...], tuple[CSEVariable, ...]], tuple[CSEVariable, ...]]rtuple[CSEVariable, ...]rwrE rr%rrErrrrrwrE)rwzdict[sympy.Symbol, sympy.Expr]rrrr(rrrrrrrr)rzOptional[CSEVariable]rwrry) rzUnion[CSEVariable, str]rrrrr z!Optional[Union[CSEVariable, str]]rwrzr#)rrrwrzrrrvrbrzrwrx)rz;Union[list[sympy.Expr], tuple[sympy.Expr, ...], sympy.Expr]rwr)rpr rr rwr)rr>rwr)%r|r}r~rrrrZrrrrrrrrrrrrrrrrrr rrrrrr$r#r1rr;rr4r5s@rPrrsM3FC9=I6=PT /( /HL /  / /D&&(,'+ F F %F % F  FF8""SW"" *"3>"FO" " """& " ; " 5 ""'" " ( " !""'"(" "  " ! ""4804 " "C "& " $ "  "1 ". "  """37 V $VV V 0 V V<""&0"9="FJ" " "4%)N'' /P/ /.,33rRrc@\rSrSr%SrS\S'SrS\S'SrS \S 'S rg) rirz ClassVar[str]rNrrrrzops_namer) r|r}r~rrrrrJrrrRrPrrs!"C"#'E 'HcrRrc^SSKnURURS9$![a gf=f)Nr) undefined)jinja2 EnvironmentStrictUndefined ImportError)rMs rP jinja2_envrQs?!!,,"   s  ,,c\rSrSrSr\S S Sjj5r\S Sj5r\SSj5rSSjr SSjr SSjr S r g )KernelTemplateiz[ Base class for defining kernel templates. Children classes: TritonTemplate, CUDATemplate cURS5n[U5S:a"USSVs/sHnSU-U-U-PM snUSS&SRU5$s snf)NTrDrFr) splitlinesrr)source num_indentsindents_spacinglinesrs rPindent_except_first"KernelTemplate.indent_except_firstsh!!$' u:>INqrIR&4<E!"Iwwu~sAc[5nUcg[RURS'SSKJn UR U5$!Uan"SSU5nU"U5UeSnAff=f)NrZr)TemplateSyntaxErrorc6^\rSrSrSU4SjjrSSjrSrU=r$)IKernelTemplate._template_from_string..DetailedTemplateSyntaxErroric>[TU]URURURUR 5 Xlgrj)r0rmessagelinenorbfilenameoriginal_error)rerdr2s rPrRKernelTemplate._template_from_string..DetailedTemplateSyntaxError.__init__s<G$&..&--&++&//  +9'rRcXSURS3nUSURS3- n[URS5(aURRR S5nUS- n[ SURS- 5n[[U5URS-5n[X45HwnXPRS- :XaSXS-S X%S3- n[URS 5(a'US S URRS- --S -- nMfMhXS-SX%S3- nMy U$)NzError in template at line  zError message: rVz Context: rrrDz: --> columnz rFz^ z: ) rbrar/rdrVsplitmaxminrrrh)re error_inforYstartendrs rPrQKernelTemplate._template_from_string..DetailedTemplateSyntaxError.__str__s:#=dkk]"!MJODLL>"DDJt22H== $ 3 3 : : @ @ F"l2 #At{{Q 7!#e*dkkAo>!&u!2A KK!O3 *QvehZr.J J #*4+>+>#I#I$.(/*-1D1D1K1Ka1O*P)Q*/)0%&J$J!+QvehZr.J J "3&%rR)rd)rdr]rwrxry)r|r}r~rrrrr4r5s@rPDetailedTemplateSyntaxErrorr_s9&&rRrp)rQrSrZfiltersrMr] from_string)rVenvr]erps rP_template_from_string$KernelTemplate._template_from_stringsjl ;-;-O-O )*.# 8??6* *"! 8 &.A &>.a0a 7C! 8sAA# AA#c0^^[RRm[U[[ 45(a0UVs0sH!oR 5UR5_M# snmO UR 5UR50mSUU4SjjnU$s snf)Nc@>TRU5nUbU$T"U5$rj)r)rbr_get_dtype_reallookups rPr1KernelTemplate._fake_get_dtype..get_dtype s'ZZ%F! "4( (rR)rbrzrwr)r5rrrnr/r0r) fake_outsrrryrzs @@rP_fake_get_dtypeKernelTemplate._fake_get_dtype s|''++ i$ / /AJK#llncmmo5KF((*I,?,?,ABF ) ) Ls(BcXlgrjrrs rPrKernelTemplate.__init__% s rRc URUR"S0UD65 g![aNn[R SU[ U5[R 5[R:S9 UsSnA$SnAff=f)z Maybe generates a new ChoiceCaller and appends it into existing choices. Returns None if success, otherwise returns the error. choices: A list of ChoiceCallers. kwargs: Additional kwargs to be passed to self.generate() to generate a new ChoiceCaller. Nz3Cannot Append Choice: %s. KernelTemplate type is %s) stack_infor) rrrrWinforogetEffectiveLevelrLINFO)rechoicesrrts rPmaybe_append_choice"KernelTemplate.maybe_append_choice( sl  NN4==262 3"  HHET 002W\\A   H s!$ A<AA71A<7A<c [e)z= Generates a ChoiceCaller instance from the given arguments. r)rers rPrKernelTemplate.generate? s "!rRrN))rVrzrWrrXrrwrz)rVrzrwr )r|zUnion[list[Buffer], Buffer]rwzCallable[[str], torch.dtype]rH)rr{rr rwzOptional[NotImplementedError])rr rwr<) r|r}r~rrrrZrur}rrrrrrRrPrSrSs >?"%8; *8*8X. %" ,/ &."rRrScL^\rSrSrSrSU4SjjrSSjrSSjrSSSjjrSSjr SSjr SSjr SSS jjr SS jr SS jrSS jrSS jrSSSjjrSrU=r$) riG c^>[TU]5 SSKJn U"5UlXlX lg)NrValueRangeAnalysis)r0rrer vr_analysisrparent_handler)rerrrr2s rPrCSEProxy.__init__J s' /-/ ,rRc^^^^^ ^ ^ UR"T/TQ70TD6m [URT5"T0TD6n[5n[ 5mSm TS:XaTS:Xa UR m OsTS:XaSTS:XaM[ RRRR[RS5R m OTS;a[UT5nU"T0TD6m TS;aT ceSm SUUU UUU U 4Sjjn[R"Xt5$) Nmaskedrr)rrr)rrrc*>[T[[45(aTT OTnT S- m TS:Xa([U[5(aURcXl[ R RR[ R RUTTS9nURTTT5 [RR(d[RR(a)Uce[[ R RX!5 U$)NrDr)rer)rnr/r0rrr5rrrrrrrrrrr) r var_dtypecsevarrprrerrbr output_idxs rPdo_cse!CSEProxy._default..do_csek s lT5M::Z(!  !OJ%Jq+$>$>177?#XX\\**  " +F  ! !$f 5##??&&>> ,,,AHH,,f@MrR)rrzrwr)_bound_variablerrr!r'rr5 interpreterrrrrrpytreetree_map) rerbrprrd dtype_handlerdtype_oprrrerrs ``` @@@@rP_defaultCSEProxy._defaultR s%%d>#''e 0 0}d3H#T4V4L ' '+ ++   >v--rRc^ SSKJn SSKJn SSKJn [ [RU5(a[R"5$[ [RU5(a[R"5$[RRm T RU:XaURRb[ URR[5(d$[!URR55eURRR#T [R"55$[$R&(az[)XA5(aj[+U 4SjS55(a[R"5$U(aeS Sjn[-[/Xr55n[1UR2U5"U6$[R"5$) z If the variable comes from an FX node, we forward the bound we have already computed Else, if the variable when codegen'ing another op, we try to compute its bounds rr)TritonTemplateKernelrD)CUDATemplateKernelc3@># UHoTR;v M g7frj)r)rr-fx_nodes rPr+CSEProxy._bound_variable.. sV0U1&0Us) set_indirectrrc[U[5(a UR$[U[R5(a [ U5$U$rj)rnrrerr.rr?s rP arg_to_bound.CSEProxy._bound_variable..arg_to_bound s8a--88O5::..&q>)HrR)r@r rwr )rerselect_algorithmrcuda.cuda_kernelrrnr5rrrrrrrdictrorrcompute_all_boundsr/rr/rrr) rerbrprrrrr arg_boundsrs @rPrCSEProxy._bound_variable sY 0;8 ahh 4 5 5&&( ( ahh 2 3 3&&( (--,, >>T !dkk&@&@&Ldkk88$??  **B ?;;--11';;N;N;PQ Q  & &73E+L+LV0UVVV"**,,  : c,56J4++T2J? ?""$$rRcl[U[5(a[R"U5n[U[R5(d[ U5U45eUR RS:GaU(a[R"U[R"U[R55nUR RS:a.[R"US5n[R"XeU5nOUn[ R""5nUR [ R""5:wa[U[R$5(aUR [![&*S5-n[!URU-URU-5nUR RS:a!UR [!S[&5-n Xy-nUR(R*R-UR(R.XWS9nUR0R3XU5n [5U5(avUR RS:+n [U[R$5(+=(d UR RU:+n UR(R7XX5 U $)Nrr)re)rnrrrr.rorerr0rbrrlongrltrrrNumberrrrrrrrr&r) rerrrrstmr new_bounds neg_boundspos sympy_var assert_lower assert_uppers rPrCSEProxy.indirect_indexing s dC ==&D$ ++?d4j$-??+ ::  a ggc3>>$ #CD::##q(QB))BS1C%,,.Jzz[0022z$ 7U7U!ZZ+vgr*BB ($$t+Z-=-=-D ::##q(**{1f'==C!+!1J++//**4;;+>+>*WC''99#UK 5 ! ! # 0 0A6L)$ ==   4BL KK $ $Yl QrRc:URRXX45$rj)rrrs rPrCSEProxy.check_bounds s{{''EAArRcXRRR;a)[RRR U5 [ U[R5(aURRX5$URRRnX;aX1$URRX5nURS:XaUR=RS- sl U$r<)rrrr5rrbrrTMPrrrrfr)rerbrrouts rPr CSEProxy.load s ;;??55 5 HH & & * *4 0 udhh / /;;,,T9 9kkoo11  $ $kkt+ ==A  KK A %  rRczX RRRU'URR(a~U[R R ;a_URRRU5nUR5H%nX RRRU'M' gggrj) rrrrr5rname_to_buffer get_output get_mutations)rerbrdr other_names rP_update_store_cacheCSEProxy._update_store_cache s,1 ##D) ;; # #0F0F(F++**55d;C!//1 :? ++J72)G #rRcURRRU5 UcURX5 U[R R ;aURRXX4S9 gg)N)rY)rrrbrr5rrrrs rPrCSEProxy.store s] &&**40 <  $ $T 1 qww.. . KK  d5  < /rRcURRRU5 URX5 U[R R ;aURRXU5$grj)rrrbrr5rrrrs rPrCSEProxy.store_reduction sV &&**40   - qww.. .;;..tEB B /rRcxUR=RS- slURRXX45$r<)rrrrs rPrCSEProxy.reduction s0 !!Q&!{{$$U~MMrRc:URRXU5$rj)rrrs rPr CSEProxy.scan! s{{F;;rRc:URRXX45$rj)rrrs rPr CSEProxy.sort, s{{CCrRc DURRUUUUUUU5$)a [Note: Inductor bucketize op] Inputs: ------- values: the values to be bucketized. boundaries: a tuple containing (a) the name of the boundaries tensor (which must be sorted, unless the sorting tensor is present), (b) the length of the tensor in the last dimension (i.e. the length of one set of boundaries), (c) the number of elements in the underlying storage (i.e. the length of the flattened tensor, ignoring striding), and (d) the stride of the tensor in the last dimension. boundary_indices: indices into a flattened version of the boundaries tensor, of the same size and shape as "values". Each index points to the first element in the set of boundaries to be used for the corresponding value. indexing_dtype: the dtype to use when indexing into the boundaries tensor. This must be int64 or int32. This additionally specifies the dtype of the return value. right: see "Details" below. sorter: an optional tuple containing (a) the name of an optional sorting tensor, used to access unsorted boundaries without reordering the boundaries tensor, and (b) the stride of the tensor in the last dimension. The values in the sorting tensor are used as indices into the *last* dimension of the boundaries tensor, with all other indices matching. The size of the sorting and boundaries tensors must be equivalent. sorter_indices: must be present if the sorting array is present; see "boundary_indices" for the equivalent definition for the boundaries tensor. Output: ------- The buckets each value belongs in, within a given set of boundaries. 0 indicates a position before the first boundary, and len(boundaries_set) represents a position after the last boundary. Details: -------- Given a value and a set of boundaries, calculate the bucket that each value belongs to. This works differently in 1-D and N-D cases. for values [[-1, 0, 1, 2], [3, 4, 5, 9]], boundaries [0, 4, 4, 8], right=True return = [[ 0, 1, 1, 1], [1, 3, 3, 4]]. for values [[-1, 0, 1, 2], [3, 4, 5, 9]], boundaries [[0, 4], [4, 8]], right=True return = [[ 0, 1, 1, 1], [0, 1, 1, 2]] Note that in the N-D boundaries case, the shape of "values" and "boundaries" must match in every dimension _except_ the last. When right == False, bucket i refers to range (boundaries[i], boundaries[i+1]]. When right == True, bucket i refers to range [boundaries[i], boundaries[i+1]). Boundaries must be non-decreasing, or a sorter must be provided which would re-index offsets in a non-decreasing order (e.g. the second output of torch.sort(offsets)). Otherwise, the result is undefined. )rrrs rPrCSEProxy.bucketize5 s1L{{$$         rR)rrr)rz Kernel[Any]rzOpsHandler[Any])rbrzrpztuple[Any, ...]rzdict[str, Any]rwr )rbrzrpr rr rwrvr ) rrrr!rrrrrwr"r#r?)rbrzrdrrwrxrjrAr@rBrDrFr'rG)r|r}r~rrbrrrrrrrrrrrrrrr4r5s@rPrrG s D-8.t+%b / /%/ /  /  /bBB&0B9=BFJB B "@SW== *=3>=FO= =CNNN& N ; N 5 N <' <  <( < ! <D'D(D D  D ! D 4804N N CN & N $ N  N 1N .N  N N rRr)rOrzrwrxr') rrzr[rr\rr]rr^Optional[CustomGraphModulePass]rwrx)rUnion[torch.device, str, None]rwzOrderedSet[BackendFeature])rrrurarwr)rrzrwzOptional[SchedulingConstructor])F)rrzryrrwr)rrzrwrrv)rSequence[sympy.Expr]rrrrrwr)rrzrrrwrx)rrzrwr)rrzrpr rr rwr)rr(rrrrrwrx)rrzrwrrr*)rwr ) __future__rr^r dataclassesenumrrrLrXrr`rer\abcrrrrrtypingr r r r r rrrrtyping_extensionsrrrrtorch.fxtorch._prims_commonr torch.utilsrrtorch.utils._ordered_setrtorch.utils._sympy.numbersrtorch.utils._sympy.printersr_PythonPrintertorch.utils._sympy.symbolrrrtorch.utils._sympy.value_rangesrrrrr dtype_propagationr! ops_handlerr"r#utilsr$r%r&r'r(r)r*r+r,r-r.r/ virtualizedr0r1r2r3r4r5collections.abcr6r7r8r9custom_graph_passr:rr;r<r=r>r"r@r#rArBrCrrFrGrrorrzr3rbr_logginggetArtifactLoggerr|rJ getLoggerrWrQ dataclassrTrrrrrrr rr KernelArgTyperrrrXrYr_rarqrvrpr{r}cachermrrrbfloat16rfloat16rrIfloat64int8int16r>rruint16r&uint64rrrrr7compile IGNORECASErrrr,r INT_TO_FLOATrrrrrrrrrrrxrr0rReductionCacheKeyr|rrrrQrSrrs0rPrs "    #   , ?)/-GOOD:;    QPBB$9>>$DD- B$hy&9%:N%JK23sELL()JF~~//*E  !=  >/ /(  C  Ta=aaH** ###   !!! === lIw8H,VW ,.).5"5"p:<6;DFAF8@D:> 7  7, 7/ 7!= 7 8 7  7 &T &3 *33$3 *35C3 3U &+   " ! V KK\U U$U U U; ;&7; ; , NNEKK MM5;;> JJ MM MM JJ KK KK KK KK LL LL LL    E    >:,' ' '' 'T: :!0:9D: :2aBaBH%N%S1S1lzz";2==Q D;#%5z#D;N  - - - 6:`6 ;HH -  `6 ;HH /-  `6;HH /- `6$;HH /- %`60;HH /- 1`6< ;HH *)   =`6L ;HH (0   M`6X ;HH 66>   Y`6h ;HH 1  i`6r ;HH 2  s`6|;HH 1 }`6F;HH 2 G`6P;HH %8$  Q`6^ ;HH &%   _`6j;HH %8  k`6v ;HH &  w`6@;HH + A`6L%;HH 88 ) M`6X%;HH 88 ) Y`6d%;HH 8 )e`6n%;HH 8 )o`6z ;HH '  {`6D ;HH (  E`6N;HH c  O`6^,;HH ? 0_`6h,;HH ? 0i`6t&;HH 9 *u`6~ ;HH *  `6H);HH D -I`6R);HH D -S`6\);HH D -]`6f);HH D -g`6p(;HH C ,q`6z$1;HH L 5${`6D$1;HH L 5$E`6N$1;HH L 5$O`6X$1;HH L 5$Y`6b';HH B +c`6l(;HH C ,m`6v(;HH C ,w`62`F -#-">*Z  NNN ,JJZ !;!;H5 +; T  k5c!1223 5e'/=0 1eP < <\3Wgo.\3~  p"p"f| ~| O8s c