8h?xSSKJr SSKJr SSKJrJr SSKrSSKJ s J r SSK J s J s Jr SSKJr SSKJr SSKJr SSKJrJrJrJrJr S \S \\S \\\\44S jrS \S\S \\S \\\S4\\S444SjrS \S \S\\ \S \\S \\\S4\\S444 Sjr!S\RDS\S \\S \\ \\ \44Sjr#S\RHS\S \\S \RH4Sjr%S\RLRNS\\(S \4Sjr)S\S\S \\S \\S44Sjr*S \RH4Sjr+g)) defaultdict)Sequence)castOptionalN) ShapeType) DeviceMesh) DTensorSpec) _StridedShardPartial Placement ReplicateShard mesh_shape placementsreturnc [U5[U5:Xd$[S[U5S[U5S35e/n[[5n[ 5n[ U5GHbupV[ U[5(a X6RRXV45 M;URXV45 [ U[5(dMdURU;a[SUSUSUS35eURU;dMURUR5 URUR5nXn[U5S:dMUR5upU RU:Xd[S U RS US 35eXU -nURU [UR545 [U5S:aM|GMe U$) a  Replace Strided Shards with regular shards in an adjusted order. Returns a list of (mesh_dim, placement) tuples where the list order is the sharding order. ex. [Shard(0), _StridedShard(0, split_factor=2), Shard(0)] -> [(0, Shard(0)), (2, Shard(0)), (1, Shard(0))] /Expected one placement per mesh dim, but found  placements and mesh dims.zTStrided sharding does not allow Shard() to appear after the strided part has ended. z at mesh dim z in z violates this assumption.rz@Can only convert _StridedShard to ordered Shard if split_factor(z) == aggregate mesh size ())len RuntimeErrorrlistset enumerate isinstancer dimappendrNotImplementedErroraddpop split_factor) rrordereddeferred_strided_placementsstrided_part_ended_for_dimmesh_dimpstrided_placementsaggregate_sizestrided_mesh_dimstrideds Y/var/www/fran/franai/venv/lib/python3.13/site-packages/torch/distributed/tensor/_utils.py_explicit_order_placementsr-s z?c*o - Z))9#j/9J+ W  G"-d"3!$ ,  a ' ' ' . 5 5xm D NNH= )!U##5566-778cxjPT%,&@B 5577.221559)D)H)H)O&%/%9N01A54F4J4J4L1(&33~E"."bcjcwcwbxy<m&6-:R&R'4'7-'GM$/I 7';<<i0AQR R304  &&r.r3c[U5S:wa [S5e[U5UR:wa%[S[U5SURS35e[ US[ 5(aU$[ US[ 5(Ga4[R"[U55n[UR55Vs/sH!n[R"X3RS9PM# nn[R"XS5 SnUSR n[UR5Vs/sH oU:wdM UPM n nUHEn [R""X9X5(d [S5eU R%5n XkU- nMG [U5n XlUSR '[R&"U 5$[S [)US5S 35es snfs snf) a Compute the global size of a DTensor from the given local tensor shape, the mesh and placements. Different from `compute_global_tensor_info`, which assumes sharding is even, this util allgathers local shards' shapes from all ranks and thus can support uneven sharding. NOTE: Currently this function only supports 1D mesh. Args: shape (:class:`torch.Size`): Shape of the local tensor mesh (:class:`DeviceMesh`): Object which describes the mesh topology of devices for the DTensor. placements (Sequence[:class:`Placement`]]): The attribute of the DTensor that describes its layout on the mesh topology. Return: tensor_shape: Shape of the global DTensor. z>compute_global_tensor_shape only supports 1 placement for now.rrrr)devicez?Non-sharded dimentions should have identical size across ranks.zPlacement type z not supported.)rrrMrrr rtorchrFrrNrI empty_likerYfuncolall_gather_inplacerequaltolistSizerO) r3r0rr>_gathered_shaped_tensorssharded_dim_sumrBd other_dims shape_tensorshape_tensor_listr/s r,compute_global_tensor_shaperhs. :!! L   :$))# Z))9$))K Q  *Q-++ JqM5 ) )ll4;/ 499;'# '   [1C1C D' #  !!"9GqM%% !&tyy!1D!1A)^a!1 D3L;;{6 8PQQ"U!- 3 3 5  ; ;O 4E{ *9Z]&&'zz,''!d:a=12/ B  '# Es(G. G ;G op_callargscfUHn[U[R[45(aURs $[U[ [ 45(dMS[U5S:dMd[US[R[45(dMUSRs $ [SUS35e)zy Find the device mesh object from args. It returns None if no mesh is found. NOTE: we can optimize this search if needed rz+Cannot find device mesh from args for op : rH) rdtensorDTensorr device_meshrr<r ValueError)rirjargs r,try_find_mesh_from_argsrq8s cGOO[9 : :?? " sT5M * *C1 3q6GOO[#ABBq6%% % B7)1M NNr. global_stridec^^S/[T5-m[U5H{up4UR5(dM[[U5R n[ [T55H-nTUTU:dMTU==URU5-ss'M/ M} [UU4Sj[ [T5555$)z Compute the stride of a local tensor shard, given the global stride of the DTensor. NOTE: Currently this function is assuming the DTensor is evenly shardable. rXc3:># UHnTUTU-v M g7f)Nr8).0rUrrstride_divisorss r, 'compute_local_stride..^s$8Q1 aOA..8Qs) rrrKrrrrNrIr<)rrr0rmesh_idxr'rUjrvs` @r,compute_local_strider{MscC ..O ,  ::<<UA""A3}-. #mA&66#A&$))H*==&/ - 8=c->P8Q r.c$[U[R5(aU$[U[5(aU/nOA[ U5S:Xa'[US[ 5(a[ US5nO [ U5n[R"U5$)z Unify variable types of size argument to torch.Size Acceptable types include: int, Sequence[int], Tuple[int], Tuple[Sequence[int]], or torch.Size rXr)rrZr`intrrr)rI torch_sizes r,normalize_to_torch_sizercsn$ ## $V TaJtAw99$q'] $Z ::j !!r.), collectionsrcollections.abcrtypingrrrZ)torch.distributed._functional_collectives distributed_functional_collectivesr\torch.distributed.tensor._apirF_apirltorch._prims_commonrtorch.distributed.device_meshr&torch.distributed.tensor._dtensor_specr (torch.distributed.tensor.placement_typesr r r r rr<r}r-r5rr2TensorrVr`rh_ops OpOverloadobjectrqr{rr8r.r,rs#$! :://)4>22'/ ':2 eCN#$2j++#-+;CI;N+ 5c?E#s(O +,+^D8D8D8DI&D8# D8  5c?E#s(O +, D8N8' LL8' *8'8@8K8' 49d3i  8'v; ::; '; 5=i5H;  ZZ; |O ZZ " "O*26*:OO*$.